labgate 0.5.40 → 0.5.43

package/README.md

@@ -1,351 +1,218 @@
 # LabGate
 
-Policy-controlled sandboxes for AI coding agents. Built for HPC clusters.
+Policy-controlled sandboxes for AI coding agents on HPC.
 
-## Current Product Focus
+## Current Focus
 
-- Primary workflow: Claude (`labgate claude`)
-- Primary runtime: Apptainer on HPC
-- macOS runtime: Podman (best-effort fallback path)
-- Secondary targets (best-effort): other agents
+- Primary workflow: `labgate claude`
+- Primary runtime: Apptainer
+- HPC default: SLURM tracking enabled by default
+- Secondary path: `labgate codex` (best-effort)
+
+Docs: [labgate.dev/docs](https://labgate.dev/docs)
 
 ## Install
 
 ```bash
 npm i -g labgate
+labgate init
 ```
 
-Note: LabGate uses `node-pty` only for the optional sticky footer. On minimal Linux installs, that dependency may fail to build without a compiler toolchain. If it fails, the install still works and LabGate falls back to non-sticky output.
+Notes:
 
-Note: `tmux` is a host-level dependency for `labgate ui` / web terminals. `npm i -g labgate` does not install `tmux`; install it through your OS or cluster module system.
+- `labgate ui` requires host `tmux`.
+- `node-pty` is optional. If it fails to build on a minimal Linux host, LabGate still works and falls back to non-sticky output.
+- Default config path is `~/.labgate/config.json`.
+- `labgate init` pre-registers a bundled sample dataset at `~/.labgate/datasets/flowers-iris`.
 
-LabGate prefers Apptainer for sandbox runtime and supports Podman as a fallback (especially on macOS).
+## Quick Start
 
-On UI startup, LabGate ensures a bundled sample dataset (`flowers-iris`) is available at `~/.labgate/datasets/flowers-iris` for first-run testing.
+Recommended two-node HPC flow:
 
-## Quick start
+1. On the login node, start the dashboard:
 
 ```bash
-labgate init                          # optional: pre-create ~/.labgate/config.json
-labgate claude                        # launch Claude Code in current dir
-labgate codex /projects/my-analysis   # launch Codex in a specific dir
-```
-
-## What it does
-
-LabGate runs your AI coding agent inside a sandboxed container with:
-
-- **Scoped filesystem** — only your working directory and configured paths are visible
-- **Credential blocking** — `.ssh`, `.aws`, `.env`, `.gnupg`, and other sensitive paths are hidden by default
-- **Network policy** — configurable network modes (`host`, `filtered`, `none`)
-- **Command blocking** — high-risk system commands (for example `mount`, `umount`, `mkfs`, `reboot`) are blocked by default
-- **Audit logging** — session start/stop and mount configuration logged to `~/.labgate/logs/`
-- **Dashboard instructions editor** — view and update per-session `AGENTS.md` / `CLAUDE.md` from the UI
-- **Session context injection** — LabGate prepends a temporary sandbox-mapping instruction block during active sessions
-- **HPC ready** — first-class Apptainer support for shared clusters
-
-## Configuration
-
-Edit `~/.labgate/config.json` to customize:
-
-```bash
-$EDITOR ~/.labgate/config.json
+labgate ui
 ```
 
-Or start fresh:
+2. On a compute node, launch Claude inside the sandbox:
 
 ```bash
-labgate init --force
+srun --pty bash
+cd /path/to/project
+labgate claude
 ```
 
-Or reset a single setting back to defaults:
+If browser auth is not practical over SSH, pass an API key directly:
 
 ```bash
-labgate config reset image
+labgate claude --api-key "$ANTHROPIC_API_KEY"
 ```
 
-### Key settings
+For local non-SSH, non-SLURM shells, `labgate claude` auto-starts `labgate ui` when the UI is not already running.
 
-| Setting | Default | What it does |
-|---------|---------|-------------|
-| `runtime` | `auto` | `auto`, `apptainer`, or `podman` |
-| `image` | `docker.io/library/node:20-bookworm` | Container image |
-| `session_timeout_hours` | `8` | Max session length |
-| `filesystem.blocked_patterns` | `.ssh, .aws, .env, ...` | Hidden from sandbox |
-| `filesystem.extra_paths` | `[]` | Additional mounts |
-| `network.mode` | `host` | `none`, `filtered`, or `host` |
-| `commands.blacklist` | `mount, umount, mkfs, reboot, shutdown` | Blocked commands |
-| `slurm.enabled` | `true` | Enable SLURM CLI passthrough (`sbatch`, `squeue`, etc.) and job tracking |
-| `headless.claude_run_with_allowed_permissions` | `true` | In Claude headless mode, auto-allow tool use via `--dangerously-skip-permissions` |
-| `headless.continuation_in_other_terminals` | `true` | Show or hide the `Continue in terminal via labgate continue ...` footer hint |
-| `headless.git_integration` | `false` | Show or hide Git integration UI (Git DAG sidebar widget and footer branch controls) |
+## What LabGate Does
 
-## Commands
+- Runs Claude or Codex inside a containerized sandbox
+- Mounts the working directory, a persistent sandbox home, configured extra paths, and named datasets
+- Hides common credential and secret paths by default
+- Applies network policy with `host` or `filtered` modes
+- Blocks high-risk commands such as `mount`, `umount`, `mkfs`, `reboot`, and `shutdown`
+- Records audit logs in `~/.labgate/logs/`
+- Tracks SLURM jobs and exposes MCP integrations for supported LabGate subsystems
+- Provides a browser UI and web-terminal control plane with `labgate ui`
+- Ships a bundled `flowers-iris` sample dataset for first-run dataset workflows
+- Lets you edit `AGENTS.md` / `CLAUDE.md` from the UI, with a temporary LabGate sandbox-context block injected for active sessions
 
-```bash
-labgate claude [workdir]    # launch Claude Code
-labgate codex [workdir]     # launch Codex
-labgate feedback            # submit feedback (interactive or piped)
-labgate status              # list running sessions
-labgate stop <id>           # stop a session
-labgate ui                  # start dashboard server on localhost:7700 (auth token required, tmux required)
-labgate register <activation-key> [--server <url>]            # activate + install enterprise license
-labgate license             # show enterprise license status
-labgate license install <key-or-file> [--system|--user|--path]  # install enterprise license key
-labgate policy init [--institution ... --admin ...]             # create policy template
-labgate policy validate [file]                                   # validate policy JSON
-labgate logs [-n 20]        # view recent audit events
-labgate logs --follow       # stream new audit events
-labgate init [--force]      # create/reset config
-labgate explore create --name <name> --repo <path> --eval "<command>"  # create Solution Explorer experiment
-labgate explore tick --experiment <id>                                  # run one autopilot tick (cron-friendly)
-labgate explore overview --experiment <id>                              # aggregated status/counts/best/latest
-labgate explore run --id <run-id>                                       # run metadata + artifact paths
-labgate explore gc --experiment <id>                                    # retention-based prune (dry-run by default)
-labgate explore compare --experiment <id> --run <run-id> --to best      # score+diff compare
-```
-
-### Solution Explorer (MVP backend)
+`network.mode=none` is rejected for `labgate claude` and `labgate codex`.
 
-LabGate now includes an early **Solution Explorer** backend for reproducible
-variant search workflows:
+## Key Defaults
 
-- Isolated experiment repo clones under `~/.labgate/explorer/repos/`
-- Git worktree-per-run execution
-- Deterministic eval contract parsing (`score` JSON on last stdout line)
-- Per-run artifacts (`eval.json`, logs, diff, summary)
-- Cron-safe autopilot tick lock
+| Setting | Default |
+| --- | --- |
+| `runtime` | `auto` |
+| `image` | `docker.io/library/node:20-bookworm` |
+| `session_timeout_hours` | `8` |
+| `network.mode` | `host` |
+| `commands.ensure_commands` | `["git"]` |
+| `slurm.enabled` | `true` |
+| `slurm.mcp_server` | `true` |
+| `audit.enabled` | `true` |
+| `headless.continuation_in_other_terminals` | `true` |
+| `headless.git_integration` | `false` |
 
-Starter template:
+Inspect or change config with:
 
 ```bash
-templates/tsp-lab/
+labgate config path
+labgate config show
+labgate config get <key>
+labgate config set <key> <value>
+labgate config reset <key>
 ```
 
-Example flow:
+## CLI Overview
 
-```bash
-# Create experiment from a local repo
-labgate explore create \
-  --name "TSP baseline" \
-  --repo /path/to/tsp-lab \
-  --eval "python3 eval.py" \
-  --agent-mode stub \
-  --stub-patch stub-patches/enable_two_opt.patch
-
-# Trigger one run
-labgate explore tick --experiment <experiment-id>
-
-# Inspect tree and leaderboard
-labgate explore tree --experiment <experiment-id> --mode best_path
-labgate explore leaderboard --experiment <experiment-id> --top 5
-```
-
-Cron example (every 5 minutes):
+Core session commands:
 
 ```bash
-*/5 * * * * /usr/bin/env labgate explore tick --experiment <experiment-id>
+labgate init
+labgate claude [workdir]
+labgate codex [workdir]
+labgate ui [--port <number> --listen-address <address> --token <string> | --socket <path>]
+labgate status
+labgate continue [idOrName] [--latest]
+labgate stop <id>
+labgate restart <id> [--dry-run]
+labgate logs [-n|--lines <count>] [--follow]
+labgate feedback [message...]
 ```
 
-### Options
+SLURM commands:
 
 ```bash
-labgate claude --dry-run              # print the sandbox command without running
-labgate claude --image my-image:tag   # use a different container image
-labgate claude --no-footer            # disable the status footer line
-labgate ui                            # localhost UI on 7700, logs full token URL + short /s/<code> quick link
-labgate ui --port 7700 --token Abc123_Xy-9Z # custom quick-link token for /s/<token>
-labgate ui --socket ~/.labgate/ui.sock # custom Unix socket path
-labgate logs --lines 50 --follow      # tail last 50 lines and keep following
+labgate slurm status [--state <state>] [--limit <count>] [--search <query>]
+labgate slurm job <id>
+labgate slurm output <id> [--stderr] [--tail <lines>]
+labgate slurm cancel <id>
+labgate slurm mcp [--db <path>]
 ```
 
-`labgate claude` auto-starts `labgate ui` when missing in local (non-SSH/non-SLURM) shells.
-
-### Shared Apptainer SIF cache (multi-user HPC)
-
-Set `LABGATE_IMAGES_DIR` to use a shared SIF cache path instead of `~/.labgate/images`:
+Dataset commands:
 
 ```bash
-export LABGATE_IMAGES_DIR=/shared/labgate/images
+labgate dataset list
+labgate dataset init <name>
 ```
 
-LabGate uses a cross-process file lock per image (`*.pull.lock`) so concurrent startups
-wait for the first pull instead of downloading the same SIF repeatedly.
-
-Optional lock tuning env vars:
-
-- `LABGATE_IMAGE_PULL_LOCK_TIMEOUT_MS` (default `3600000`)
-- `LABGATE_IMAGE_PULL_LOCK_STALE_MS` (default `7200000`)
-- `LABGATE_IMAGE_PULL_LOCK_POLL_MS` (default `750`)
+`labgate dataset init` scans an already-registered dataset entry and stores file count and size metadata in config.
 
-### SLURM inside sandboxes (`sbatch` / `squeue`)
-
-For Apptainer sessions, LabGate now attempts SLURM CLI passthrough automatically.
-If host `sbatch`/`squeue` are available, they are staged into the sandbox, so
-`labgate claude` should work without extra config in the common HPC path.
-
-SLURM tracking and MCP tools are enabled by default (`slurm.enabled=true`).
-If native SQLite (`better-sqlite3`) is unavailable on a host, LabGate falls back
-to a JSON tracking store automatically.
-
-Requirements for automatic `sbatch` in sandbox:
-
-1. Runtime is Apptainer
-2. The host can resolve SLURM CLI tools when launching LabGate
-
-If `sbatch` is missing inside the sandbox, run:
+Solution Explorer commands:
 
 ```bash
-which sbatch                          # on host, before launching labgate
-labgate claude
+labgate explore create --name <name> --repo <path> --eval <command> [options]
+labgate explore list [--limit <count>] [--offset <count>]
+labgate explore status <experimentId> [-n|--limit <count>]
+labgate explore pause <experimentId>
+labgate explore resume <experimentId>
+labgate explore tick --experiment <id>
+labgate explore tree --experiment <id> [--mode best_path|full]
+labgate explore leaderboard --experiment <id> [-k|--top <count>]
+labgate explore gc --experiment <id> [--yes]
+labgate explore retention show --experiment <id>
+labgate explore retention set --experiment <id> [retention flags]
+labgate explore compare --experiment <id> --run <runId> [--to best|parent|<runId>] [--diff]
+labgate explore overview --experiment <id>
+labgate explore run --id <runId>
+labgate explore mcp [--db <path>]
 ```
 
-If your cluster uses environment modules, load SLURM first (host shell), then launch LabGate:
+For Claude/Codex sessions, LabGate can also register dataset, cluster, and SLURM MCP servers inside the sandbox when the relevant integrations are enabled.
 
-```bash
-module load slurm   # or your site-specific module name
-labgate claude
-```
-
-Optional (disable SLURM tracking DB + MCP server):
-
-```bash
-labgate config set slurm.enabled false
-```
-
-## Feedback
-
-Submit feedback from the CLI:
+Enterprise commands:
 
 ```bash
-labgate feedback
-echo "This was great" | labgate feedback
-labgate feedback "Short feedback message"
+labgate license
+labgate license install <keyOrFile> [--system|--user|--path] [--overwrite]
+labgate register <activationKey> [--server <url>] [--token <token>] [--timeout <ms>] [--system|--user|--path] [--overwrite]
+labgate policy validate [file]
+labgate policy init [--path <path>] [--institution <name>] [--admin <username>] [--runtime <runtime>] [--force]
 ```
 
-If `LABGATE_FEEDBACK_URL` is set, LabGate will `POST` feedback JSON to that URL.
-If `LABGATE_FEEDBACK_TOKEN` is set, it will be sent as a Bearer token.
-If no URL is configured, LabGate defaults to `https://labgate.dev/api/feedback`.
-If the request fails (or `LABGATE_FEEDBACK_DISABLE=1`), feedback is saved locally at `~/.labgate/feedback.jsonl`.
+For full options, use `labgate <command> --help` or the docs site.
 
-## Testing
+## Apptainer and SLURM Notes
 
-Run tests:
+- The primary supported path is login-node `labgate ui` plus compute-node `labgate claude`.
+- LabGate prefers Apptainer on HPC. If you manage runtime explicitly, prefer `apptainer`.
+- SLURM tracking is enabled by default.
+- For Apptainer sessions, LabGate expects host SLURM CLIs such as `sbatch` and `squeue` to be available when the session starts.
+- If your site uses environment modules, load SLURM before launching LabGate:
 
 ```bash
-npm run setup             # install dependencies
-npm run verify:quick      # build + unit tests
-npm run verify            # build + unit + integration tests
-npm run dev:claude        # start UI in background, then launch local labgate claude
-npm run test:unit         # fast unit tests
-npm run test:integration  # dashboard integration tests
-npm run test:e2e:real     # opt-in real runtime OAuth/browser smoke test
-npm test                  # unit + integration
-npm run release:check     # verify + npm pack --dry-run
+module load slurm
+labgate claude
 ```
 
-### Corporate mode smoke test (local or HPC)
+- In SLURM job scripts, use relative paths or real host paths for `#SBATCH --output` and `#SBATCH --error`, not container-only paths such as `/work/...`.
+
+Shared SIF cache:
 
 ```bash
-# 1) Create a signed enterprise key (issuer side)
-export LABGATE_LICENSE_SECRET='replace-with-your-signing-secret'
-LICENSE_KEY="$(npx tsx scripts/generate-license.ts \
-  --institution 'Example University' \
-  --tier pro \
-  --expires 2099-12-31 2>/dev/null)"
-
-# 2) Install key on target host (admin side)
-labgate license install "$LICENSE_KEY" --path /tmp/labgate/license.key --overwrite
-# HPC system-wide install (root):
-# sudo labgate license install "$LICENSE_KEY" --system --overwrite
-
-# Optional online activation (instead of direct install):
-# Uses default endpoint: https://labgate.dev/api/license/activate
-# labgate register '<activation-key-from-vendor>' --path /tmp/labgate/license.key --overwrite
-# Optional custom endpoint:
-# export LABGATE_ACTIVATION_URL='https://your-control-plane.example.com/api/license/activate'
-
-# 3) Bootstrap and validate policy
-labgate policy init --path /tmp/labgate/policy.json --admin "$(whoami)" --force
-labgate policy validate /tmp/labgate/policy.json
-
-# 4) Verify forced settings are locked for users
-LABGATE_LICENSE_PATH=/tmp/labgate/license.key \
-LABGATE_POLICY_PATH=/tmp/labgate/policy.json \
-labgate config set runtime auto
-# expected: error about admin-locked field
-
-# 5) Open dashboard and verify UI lock labels ("set by admin")
-LABGATE_LICENSE_PATH=/tmp/labgate/license.key \
-LABGATE_POLICY_PATH=/tmp/labgate/policy.json \
-labgate ui
+labgate config set images_dir /shared/labgate/images
 ```
 
-Automated enterprise coverage:
+Or with an environment override:
 
 ```bash
-npx vitest run -c vitest.integration.config.ts src/lib/cli.enterprise.integration.test.ts src/lib/ui.integration.test.ts
+export LABGATE_IMAGES_DIR=/shared/labgate/images
 ```
 
-CI automation runs `npm run verify` on every push to `main` and on pull requests (`.github/workflows/ci.yml`).
-`npm run test:integration` automatically rebuilds `better-sqlite3` first to avoid Node ABI mismatch errors after Node upgrades.
+`LABGATE_IMAGES_DIR` takes precedence over `images_dir`.
 
-Dev launcher options:
+## Feedback
 
 ```bash
-LABGATE_UI_PORT=7711 npm run dev:claude -- /path/to/workdir   # custom UI port + workdir
-LABGATE_KEEP_UI=1 npm run dev:claude -- .                     # keep UI running after claude exits
+labgate feedback
+labgate feedback "Short feedback message"
+echo "This was great" | labgate feedback
 ```
 
-Coverage:
-
-- **`npm run test:unit`** covers config/runtime/container helpers
-- **`npm run test:integration`** covers dashboard flow that:
-  1. Builds the CLI
-  2. Starts `labgate ui` on a temporary localhost port
-  3. Launches `labgate claude` and `labgate codex` with a mocked runtime
-  4. Verifies sessions appear in `/api/sessions` (dashboard data source)
-  5. Stops a session through `POST /api/sessions/stop` and verifies it disappears
-  6. Verifies UI port fallback when requested ports are occupied
-  7. Verifies `/api/config` accepts valid payloads and rejects invalid payloads without mutating config
-  8. Verifies `labgate logs --follow` prints tail output and streams newly appended events
-  9. Verifies dashboard activity inference for accessed files and websites from agent transcripts
-  10. Verifies `GET/PUT /api/sessions/:id/instructions` with conflict detection for `AGENTS.md` and `CLAUDE.md`
-- **`npm run test:e2e:real`** runs a real runtime smoke test for OAuth/browser opening:
-  1. Launches real `labgate claude` (no mocked container runtime)
-  2. Waits for OAuth URL flow
-  3. Verifies host browser-open hook is triggered
-  4. Optional override: `LABGATE_REAL_E2E_IMAGE`
+Feedback posts to `LABGATE_FEEDBACK_URL` when set, or to `https://labgate.dev/api/feedback` by default. If remote submission fails, LabGate saves feedback locally to `~/.labgate/feedback.jsonl`.
 
-## How it works
-
-LabGate builds a sandboxed container from your config:
-
-1. Detects Apptainer first, then Podman (or uses explicit runtime)
-2. Mounts your working directory at `/work`
-3. Mounts persistent sandbox HOME at `/home/sandbox` (for npm cache, agent config)
-4. Overlays blocked paths (`.ssh`, `.aws`, etc.) with empty mounts
-5. Applies network isolation and capability restrictions
-6. Installs the agent (if not cached) and runs it interactively
-
-On macOS, LabGate syncs your Claude credentials from the system keychain so the agent can authenticate automatically.
-
-## Audit logs
-
-Session events are logged to `~/.labgate/logs/YYYY-MM-DD.jsonl`:
+## Development
 
 ```bash
-cat ~/.labgate/logs/2025-02-05.jsonl | jq .
+npm run setup
+npm run verify:quick
+npm run verify
+npm run test:unit
+npm run test:integration
+npm run test:e2e:real
+npm run dev:claude
+npm run release:check
 ```
 
-## Roadmap
-
-- **M0** CLI + sandbox engine + config + audit (this release)
-- **M1** Mount allowlists, network filtering, project-level config
-- **M2** SLURM proxy (submit/status/cancel from inside sandbox)
-- **M3** Web UI for config + audit viewer
-- **M4** Institutional mode (/etc/labgate/ policies, admin locks)
-
 ## License
 
-MIT
+License terms: [labgate.dev](https://labgate.dev)

package/bin/postinstall.js

@@ -0,0 +1,40 @@
+'use strict';
+
+function shouldShowGlobalInstallHint(env = process.env) {
+  if (env.LABGATE_SKIP_POSTINSTALL_HINT === '1') {
+    return false;
+  }
+  return env.npm_config_global === 'true' || env.npm_config_location === 'global';
+}
+
+function buildPostInstallMessage() {
+  return [
+    'LabGate is ready.',
+    '',
+    'Next steps:',
+    '  cd /path/to/project',
+    '  labgate claude   # recommended: start Claude in a sandbox',
+    '  labgate codex    # start Codex in a sandbox',
+    '  labgate ui       # open the local dashboard',
+    '',
+    'LabGate creates ~/.labgate/config.json automatically on first run.',
+    'Docs: https://labgate.dev/quickstart',
+  ].join('\n');
+}
+
+function main({ env = process.env, stdout = process.stdout } = {}) {
+  if (!shouldShowGlobalInstallHint(env)) {
+    return;
+  }
+  stdout.write(`\n${buildPostInstallMessage()}\n`);
+}
+
+if (require.main === module) {
+  main();
+}
+
+module.exports = {
+  buildPostInstallMessage,
+  main,
+  shouldShowGlobalInstallHint,
+};