labgate 0.5.26 → 0.5.28

package/README.md

@@ -1,13 +1,23 @@
 # LabGate
 
-Policy-controlled sandboxes for AI coding agents. Built for HPC clusters.
+Secure wrapper around LLM coding agents for HPC clusters.
+
+LabGate lets institutions adopt AI coding tools without giving agents unrestricted host access. It is designed for shared research environments where HPC admins need policy and audit controls, while researchers need a practical day-to-day workflow for coding, data analysis, and SLURM jobs.
+
+## Product Goal
+
+- Give HPC admins a deployable control layer for agent sessions.
+- Make Claude-assisted work practical for researchers on real cluster infrastructure.
+- Keep the default path simple and reliable: `labgate claude` + Apptainer + SLURM.
 
 ## Current Product Focus
 
 - Primary workflow: Claude (`labgate claude`)
 - Primary runtime: Apptainer on HPC
-- macOS runtime: Podman (best-effort fallback path)
-- Secondary targets (best-effort): other agents
+- SLURM integration: enabled by default (`slurm.enabled = true`)
+- Secondary targets: other agents/runtimes are best-effort only
+
+LabGate still contains Podman runtime code for local/non-HPC scenarios, but that is not the primary supported path.
 
 ## Install
 
@@ -15,46 +25,107 @@ Policy-controlled sandboxes for AI coding agents. Built for HPC clusters.
 npm i -g labgate
 ```
 
-Note: LabGate uses `node-pty` only for the optional sticky footer. On minimal Linux installs, that dependency may fail to build without a compiler toolchain. If it fails, the install still works and LabGate falls back to non-sticky output.
+Note: LabGate uses `node-pty` only for the optional sticky footer. On minimal Linux installs, that dependency may fail to build without a compiler toolchain. If it fails, install still succeeds and LabGate falls back to non-sticky output.
+
+## Quick Start (Researcher)
+
+```bash
+labgate init
+labgate claude
+```
+
+Typical HPC flow:
+
+1. Login node: run `labgate ui`
+2. Compute allocation: `srun --pty bash`, then `labgate claude` in your project directory
+
+Useful follow-ups for data-heavy work:
+
+```bash
+labgate dataset list
+labgate slurm status
+labgate logs --follow
+```
+
+Example (life science / data analysis workflow):
 
-LabGate prefers Apptainer for sandbox runtime and supports Podman as a fallback (especially on macOS).
+```bash
+# 1) register datasets in ~/.labgate/config.json (via UI or config edit)
+# 2) initialize dataset stats for discoverability
+labgate dataset init rnaseq-cohort
+
+# 3) start agent and run analysis in project directory
+labgate claude
+
+# 4) submit SLURM job with host-valid output paths (relative preferred)
+sbatch --output slurm-%j.out --error slurm-%j.err run_qc.sh
 
-## Quick start
+# 5) inspect tracked jobs and output
+labgate slurm status
+labgate slurm output <job-id> --tail 100
+```
+
+## Quick Start (HPC Admin)
 
 ```bash
-labgate init                          # create ~/.labgate/config.json
-labgate claude                        # launch Claude Code in current dir
-labgate codex /projects/my-analysis   # launch Codex in a specific dir
+# Install license (enterprise mode)
+labgate license install <key-or-file> --system
+
+# Create baseline policy
+labgate policy init --path /etc/labgate/policy.json --admin <hpc-admin-username>
+labgate policy validate
+
+# Validate default runtime behavior
+labgate config get runtime
 ```
 
-## What it does
+Admin controls can force/lock runtime, network mode, audit settings, and SLURM behavior through policy.
 
-LabGate runs your AI coding agent inside a sandboxed container with:
+## Why HPC Admins Deploy It
 
-- **Scoped filesystem** — only your working directory and configured paths are visible
-- **Credential blocking** — `.ssh`, `.aws`, `.env`, `.gnupg`, and other sensitive paths are hidden by default
-- **Network policy** — configurable network modes (`host`, `filtered`, `none`)
-- **Command blocking** — `ssh`, `curl`, `wget`, and other commands are blocked by default
-- **Audit logging** — session start/stop and mount configuration logged to `~/.labgate/logs/`
-- **Dashboard instructions editor** — view and update per-session `AGENTS.md` / `CLAUDE.md` from the UI
-- **Session context injection** — LabGate prepends a temporary sandbox-mapping instruction block during active sessions
-- **HPC ready** — first-class Apptainer support for shared clusters
+- Scoped filesystem mounts instead of full host exposure
+- Default blocking of common credential and key material paths (`.ssh`, `.aws`, `.env`, `.gnupg`, key files)
+- Network policy modes (`host`, `filtered`, `none`)
+- Command blacklist inside sandbox (`ssh`, `curl`, `wget`, etc.)
+- Session/audit logging for operational traceability
+- Enterprise policy and lock semantics for institution-level governance
+- SLURM-aware behavior designed for shared cluster operations
+
+## Why Researchers Keep Using It (Life Science / Data Analysis)
+
+- Works in existing project folders and scheduler workflows
+- Named dataset mounts under `/datasets/<name>` reduce path confusion in collaborative analysis
+- Auto-injected session context gives the agent correct path + cluster constraints
+- SLURM tracking + MCP tools help inspect jobs and output without leaving the coding workflow
+- Results registry MCP lets teams record findings, artifacts, and summaries across sessions
+
+## What LabGate Enforces
+
+LabGate runs AI coding agents in a sandboxed container with:
+
+- **Scoped filesystem**: only workdir + configured mounts are visible
+- **Credential blocking**: sensitive paths hidden by default
+- **Network policy**: configurable network mode
+- **Command blocking**: risky commands blocked by default
+- **Audit logging**: session lifecycle + key security events in `~/.labgate/logs/`
+- **Instruction management**: temporary LabGate context blocks in `CLAUDE.md` / `AGENTS.md`
+- **HPC integration**: Apptainer-first runtime behavior and SLURM support
 
 ## Configuration
 
-Edit `~/.labgate/config.json` to customize:
+Edit config:
 
 ```bash
 $EDITOR ~/.labgate/config.json
 ```
 
-Or start fresh:
+Reset full config:
 
 ```bash
 labgate init --force
 ```
 
-Or reset a single setting back to defaults:
+Reset a single setting to defaults:
 
 ```bash
 labgate config reset image
@@ -64,71 +135,101 @@ labgate config reset image
 
 | Setting | Default | What it does |
 |---------|---------|-------------|
-| `runtime` | `auto` | `auto`, `apptainer`, or `podman` |
+| `runtime` | `auto` | Runtime preference (`auto`, `apptainer`, `podman`) |
 | `image` | `docker.io/library/node:20-bookworm` | Container image |
 | `session_timeout_hours` | `8` | Max session length |
 | `filesystem.blocked_patterns` | `.ssh, .aws, .env, ...` | Hidden from sandbox |
 | `filesystem.extra_paths` | `[]` | Additional mounts |
+| `datasets` | `[]` | Named dataset mounts under `/datasets/*` |
 | `network.mode` | `host` | `none`, `filtered`, or `host` |
 | `commands.blacklist` | `ssh, curl, wget, ...` | Blocked commands |
-| `slurm.enabled` | `true` | Enable SLURM CLI passthrough (`sbatch`, `squeue`, etc.) and job tracking |
+| `slurm.enabled` | `true` | Enable SLURM tracking + passthrough |
+| `slurm.mcp_server` | `true` | Enable SLURM MCP server integration |
+| `audit.enabled` | `true` | Enable audit logging |
 
 ## Commands
 
 ```bash
-labgate claude [workdir]    # launch Claude Code
-labgate codex [workdir]     # launch Codex
-labgate feedback            # submit feedback (interactive or piped)
-labgate status              # list running sessions
-labgate stop <id>           # stop a session
-labgate ui                  # start dashboard server (owner-only Unix socket by default)
-labgate register <activation-key> [--server <url>]            # activate + install enterprise license
-labgate license             # show enterprise license status
-labgate license install <key-or-file> [--system|--user|--path]  # install enterprise license key
-labgate policy init [--institution ... --admin ...]             # create policy template
-labgate policy validate [file]                                   # validate policy JSON
-labgate logs [-n 20]        # view recent audit events
-labgate logs --follow       # stream new audit events
-labgate init [--force]      # create/reset config
+# Agent sessions
+labgate claude [workdir]
+labgate codex [workdir]              # secondary/best-effort path
+
+# Session lifecycle
+labgate status
+labgate stop <id>
+labgate restart <id>
+labgate continue [web-terminal-id] [--latest]
+
+# UI + logs
+labgate ui
+labgate logs [-n 20]
+labgate logs --follow
+labgate doctor
+
+# Config + setup
+labgate init [--force]
+labgate config get <key>
+labgate config set <key> <value>
+labgate config reset <key>
+
+# Dataset workflow
+labgate dataset list
+labgate dataset init <name>
+
+# SLURM workflow
+labgate slurm status
+labgate slurm job <id>
+labgate slurm output <id> [--stderr] [--tail <lines>]
+labgate slurm cancel <id>
+labgate slurm mcp
+
+# Enterprise
+labgate license
+labgate license install <key-or-file> [--system|--user|--path]
+labgate register <activation-key> [--server <url>]
+labgate policy init [--institution ... --admin ...]
+labgate policy validate [file]
 ```
 
-### Options
+### Common options
 
 ```bash
-labgate claude --dry-run              # print the sandbox command without running
-labgate claude --image my-image:tag   # use a different container image
-labgate claude --no-footer            # disable the status footer line
-labgate ui --port 7700                # localhost UI, logs full token URL + short /s/<code> quick link
-labgate ui --socket ~/.labgate/ui.sock # custom Unix socket path
-labgate logs --lines 50 --follow      # tail last 50 lines and keep following
+labgate claude --dry-run
+labgate claude --image my-image:tag
+labgate claude --no-footer
+labgate claude --api-key "$ANTHROPIC_API_KEY"
+labgate ui --socket ~/.labgate/ui.sock
+labgate logs --lines 50 --follow
 ```
 
+`labgate claude` auto-starts `labgate ui` when missing in local (non-SSH/non-SLURM) shells.
+
 ### SLURM inside sandboxes (`sbatch` / `squeue`)
 
-For Apptainer sessions, LabGate now attempts SLURM CLI passthrough automatically.
-If host `sbatch`/`squeue` are available, they are staged into the sandbox, so
-`labgate claude` should work without extra config in the common HPC path.
+For Apptainer sessions, LabGate attempts SLURM CLI passthrough automatically.
+If host `sbatch`/`squeue` are available, they are staged into the sandbox so
+`labgate claude` works in common HPC setups without extra config.
 
-SLURM tracking and MCP tools are enabled by default (`slurm.enabled=true`).
+SLURM tracking and MCP tools are enabled by default (`slurm.enabled = true`).
 If native SQLite (`better-sqlite3`) is unavailable on a host, LabGate falls back
 to a JSON tracking store automatically.
 
 Requirements for automatic `sbatch` in sandbox:
 
 1. Runtime is Apptainer
-2. The host can resolve SLURM CLI tools when launching LabGate
+2. Host shell can resolve SLURM CLI tools before launching LabGate
 
-If `sbatch` is missing inside the sandbox, run:
+If `sbatch` is missing inside the sandbox:
 
 ```bash
-which sbatch                          # on host, before launching labgate
+which sbatch
 labgate claude
 ```
 
-If your cluster uses environment modules, load SLURM first (host shell), then launch LabGate:
+If your cluster uses environment modules, load SLURM first:
 
 ```bash
-module load slurm   # or your site-specific module name
+module load slurm
 labgate claude
 ```
 
@@ -203,7 +304,7 @@ labgate config set runtime auto
 # 5) Open dashboard and verify UI lock labels ("set by admin")
 LABGATE_LICENSE_PATH=/tmp/labgate/license.key \
 LABGATE_POLICY_PATH=/tmp/labgate/policy.json \
-labgate ui --port 7700
+labgate ui
 ```
 
 Automated enterprise coverage:
@@ -242,20 +343,20 @@ Coverage:
   3. Verifies host browser-open hook is triggered
   4. Optional override: `LABGATE_REAL_E2E_IMAGE`
 
-## How it works
+## How It Works
 
 LabGate builds a sandboxed container from your config:
 
-1. Detects Apptainer first, then Podman (or uses explicit runtime)
+1. Detects Apptainer first (primary HPC path), with secondary fallback runtimes when configured
 2. Mounts your working directory at `/work`
 3. Mounts persistent sandbox HOME at `/home/sandbox` (for npm cache, agent config)
 4. Overlays blocked paths (`.ssh`, `.aws`, etc.) with empty mounts
-5. Applies network isolation and capability restrictions
+5. Applies network isolation and command controls
 6. Installs the agent (if not cached) and runs it interactively
 
-On macOS, LabGate syncs your Claude credentials from the system keychain so the agent can authenticate automatically.
+On macOS, LabGate can sync Claude credentials from the system keychain so the agent can authenticate automatically.
 
-## Audit logs
+## Audit Logs
 
 Session events are logged to `~/.labgate/logs/YYYY-MM-DD.jsonl`:
 
@@ -263,14 +364,6 @@ Session events are logged to `~/.labgate/logs/YYYY-MM-DD.jsonl`:
 cat ~/.labgate/logs/2025-02-05.jsonl | jq .
 ```
 
-## Roadmap
-
-- **M0** CLI + sandbox engine + config + audit (this release)
-- **M1** Mount allowlists, network filtering, project-level config
-- **M2** SLURM proxy (submit/status/cancel from inside sandbox)
-- **M3** Web UI for config + audit viewer
-- **M4** Institutional mode (/etc/labgate/ policies, admin locks)
-
 ## License
 
 MIT