@kradle/cli 0.2.0 → 0.2.2

package/static/ai_docs/LLM_CLI_REFERENCE.md

@@ -0,0 +1,669 @@
+# Kradle CLI - LLM Reference Guide
+
+This document provides exhaustive documentation for LLM agents using the Kradle CLI to manage Minecraft challenges, experiments, and agents.
+
+## Table of Contents
+
+- [Prerequisites](#prerequisites)
+- [Configuration](#configuration)
+- [Project Initialization](#project-initialization)
+- [Challenge Commands](#challenge-commands)
+- [Experiment Commands](#experiment-commands)
+- [Agent Commands](#agent-commands)
+- [AI Docs Commands](#ai-docs-commands)
+- [Common Workflows](#common-workflows)
+- [Error Handling](#error-handling)
+
+---
+
+## Prerequisites
+
+Before using the CLI, ensure:
+1. The CLI is installed globally: `npm i -g @kradle/cli`
+2. You are in an initialized Kradle project directory (has `.env` file with API key)
+3. You have a valid `KRADLE_API_KEY` from https://kradle.ai/settings#api-keys
+
+---
+
+## Configuration
+
+The CLI reads configuration from environment variables or command-line flags. Flags override environment variables.
+
+### Environment Variables
+
+| Variable | Description | Required |
+|----------|-------------|----------|
+| `KRADLE_API_KEY` | Your Kradle API key | Yes |
+| `KRADLE_API_URL` | API endpoint (default: `https://api.kradle.ai/v0`) | No |
+| `KRADLE_WEB_URL` | Web URL (default: `https://kradle.ai`) | No |
+| `KRADLE_CHALLENGES_PATH` | Path to built datapacks | No |
+| `NAMESPACE` | Default namespace for challenges | No |
+
+### Global Flags
+
+These flags are available on most commands:
+
+| Flag | Environment Variable | Description |
+|------|---------------------|-------------|
+| `--api-key` | `KRADLE_API_KEY` | Kradle API key |
+| `--api-url` | `KRADLE_API_URL` | Kradle API URL |
+| `--web-url` | `KRADLE_WEB_URL` | Kradle Web URL |
+
+---
+
+## Project Initialization
+
+### `kradle init`
+
+Initializes a new Kradle project in the current directory.
+
+**Usage:**
+```bash
+kradle init
+```
+
+**Interactive Prompts:**
+1. API key (if not set in environment)
+2. Environment selection (dev/prod)
+
+**Created Files:**
+- `.env` - Environment configuration with API key
+- `package.json` - Node.js package configuration
+- `tsconfig.json` - TypeScript configuration
+- `.gitignore` - Git ignore rules
+- `template-run.json` - Run configuration template
+
+**Example:**
+```bash
+mkdir my-kradle-project
+cd my-kradle-project
+kradle init
+# Follow prompts to enter API key and select environment
+```
+
+---
+
+## Challenge Commands
+
+Challenges are Minecraft scenarios that agents can be tested against. Each challenge consists of:
+- `challenge.ts` - Defines the challenge behavior using the Sandstone API
+- `config.ts` - Metadata (name, roles, objectives, visibility)
+
+### `kradle challenge create <name>`
+
+Creates a new challenge locally and in the cloud.
+
+**Usage:**
+```bash
+kradle challenge create <challenge-name>
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `challenge-name` | Unique identifier for the challenge (lowercase, hyphens allowed) | Yes |
+
+**What it does:**
+1. Creates `challenges/<name>/challenge.ts` from template
+2. Registers the challenge in the cloud
+3. Downloads and creates `challenges/<name>/config.ts` with cloud-generated metadata
+
+**Example:**
+```bash
+kradle challenge create my-first-challenge
+# Creates challenges/my-first-challenge/challenge.ts
+# Creates challenges/my-first-challenge/config.ts
+```
+
+---
+
+### `kradle challenge build <name>`
+
+Builds a challenge datapack and uploads everything to the cloud.
+
+**Usage:**
+```bash
+kradle challenge build <challenge-name>
+kradle challenge build <challenge-name> --visibility public
+kradle challenge build --all
+kradle challenge build --all --visibility public
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `challenge-name` | Challenge to build | Yes (unless `--all`) |
+
+**Flags:**
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--all` | Build all local challenges | false |
+| `--visibility` | Set visibility (`private` or `public`) | private |
+
+**What it does:**
+1. Creates the challenge in cloud if it doesn't exist
+2. Uploads `config.ts` metadata to cloud
+3. Executes `challenge.ts` to generate the datapack
+4. Compresses and uploads datapack to cloud storage
+
+**Examples:**
+```bash
+# Build single challenge
+kradle challenge build my-challenge
+
+# Build and make public
+kradle challenge build my-challenge --visibility public
+
+# Build all challenges
+kradle challenge build --all
+```
+
+---
+
+### `kradle challenge delete <name>`
+
+Deletes a challenge locally, from the cloud, or both.
+
+**Usage:**
+```bash
+kradle challenge delete <challenge-name>
+kradle challenge delete <challenge-name> --yes
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `challenge-name` | Challenge to delete | Yes |
+
+**Flags:**
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--yes`, `-y` | Skip confirmation prompts | false |
+
+**Interactive prompts (unless `--yes`):**
+1. Delete local files? (y/n)
+2. Delete from cloud? (y/n)
+
+**Examples:**
+```bash
+# Interactive deletion
+kradle challenge delete my-challenge
+
+# Force delete without prompts
+kradle challenge delete my-challenge --yes
+```
+
+---
+
+### `kradle challenge list`
+
+Lists all challenges (both local and cloud).
+
+**Usage:**
+```bash
+kradle challenge list
+```
+
+**Output format:**
+```
+Local challenges:
+  - my-challenge
+  - another-challenge
+
+Cloud challenges:
+  - my-challenge (public)
+  - team-kradle:example-challenge (public)
+  - my-private-challenge (private)
+```
+
+**Example:**
+```bash
+kradle challenge list
+```
+
+---
+
+### `kradle challenge watch <name>`
+
+Watches a challenge for file changes and automatically rebuilds/uploads.
+
+**Usage:**
+```bash
+kradle challenge watch <challenge-name>
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `challenge-name` | Challenge to watch | Yes |
+
+**Behavior:**
+- Monitors `challenges/<name>/` directory for changes
+- Debounces rebuilds (300ms delay)
+- Only uploads if datapack hash changed
+- Press Ctrl+C to stop watching
+
+**Example:**
+```bash
+kradle challenge watch my-challenge
+# Now edit challenges/my-challenge/challenge.ts
+# Changes will automatically rebuild and upload
+```
+
+---
+
+### `kradle challenge run <name>`
+
+Runs a challenge with configured participants.
+
+**Usage:**
+```bash
+kradle challenge run <challenge-name>
+kradle challenge run <challenge-name> --studio
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `challenge-name` | Challenge to run | Yes |
+
+**Flags:**
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--studio` | Run in local studio environment instead of production | false |
+
+**Prerequisites:**
+- `template-run.json` must exist in project root with participant configuration
+
+**Example `template-run.json`:**
+```json
+{
+  "participants": [
+    {
+      "id": "agent-1",
+      "type": "agent",
+      "agentId": "team-kradle:my-agent"
+    },
+    {
+      "id": "player-1",
+      "type": "human"
+    }
+  ]
+}
+```
+
+**Examples:**
+```bash
+# Run in production
+kradle challenge run my-challenge
+
+# Run in local studio
+kradle challenge run my-challenge --studio
+```
+
+---
+
+## Experiment Commands
+
+Experiments allow batch execution of challenge runs with different configurations for benchmarking and analysis.
+
+### Concepts
+
+**Experiment:** A named collection of run configurations in `experiments/<name>/config.ts`.
+
+**Version:** A snapshot of experiment execution stored in `experiments/<name>/versions/001/`, `002/`, etc. Contains:
+- Copy of `config.ts` at execution time
+- `manifest.json` - Generated list of runs
+- `progress.json` - Status of each run
+- `recordings/` - Downloaded gameplay recordings (optional)
+
+---
+
+### `kradle experiment create <name>`
+
+Creates a new experiment with a template configuration.
+
+**Usage:**
+```bash
+kradle experiment create <experiment-name>
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `experiment-name` | Unique name for the experiment | Yes |
+
+**Interactive prompts:**
+1. Select a challenge to use (from available challenges)
+
+**Created files:**
+- `experiments/<name>/config.ts` - Experiment configuration template
+
+**Example `config.ts` structure:**
+```typescript
+export async function main() {
+  return {
+    runs: [
+      {
+        challenge: "my-challenge",
+        participants: [
+          { id: "agent-1", type: "agent", agentId: "team-kradle:my-agent" }
+        ]
+      }
+    ],
+    tags: ["experiment-tag"]
+  };
+}
+```
+
+**Example:**
+```bash
+kradle experiment create benchmark-v1
+# Select challenge from list
+# Edit experiments/benchmark-v1/config.ts to configure runs
+```
+
+---
+
+### `kradle experiment run <name>`
+
+Executes or resumes an experiment.
+
+**Usage:**
+```bash
+kradle experiment run <name>
+kradle experiment run <name> --new-version
+kradle experiment run <name> --max-concurrent 10
+kradle experiment run <name> --download-recordings
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `name` | Experiment name | Yes |
+
+**Flags:**
+| Flag | Short | Description | Default |
+|------|-------|-------------|---------|
+| `--new-version` | `-n` | Start a fresh version instead of resuming | false |
+| `--max-concurrent` | `-m` | Maximum parallel runs | 5 |
+| `--download-recordings` | `-d` | Auto-download recordings as runs complete | false |
+
+**Behavior:**
+1. Creates new version or resumes existing incomplete version
+2. Executes `config.ts` to generate run manifest
+3. Displays interactive TUI showing run progress
+4. Saves progress periodically (supports interruption and resume)
+5. Opens Metabase dashboard when complete
+
+**Examples:**
+```bash
+# Resume or start experiment
+kradle experiment run benchmark-v1
+
+# Force new version
+kradle experiment run benchmark-v1 --new-version
+
+# Run with higher parallelism
+kradle experiment run benchmark-v1 --max-concurrent 10
+
+# Auto-download recordings
+kradle experiment run benchmark-v1 --download-recordings
+
+# Combine flags
+kradle experiment run benchmark-v1 -n -m 10 -d
+```
+
+---
+
+### `kradle experiment recordings <name>`
+
+Downloads gameplay recordings from completed experiment runs.
+
+**Usage:**
+```bash
+kradle experiment recordings <name>
+kradle experiment recordings <name> <run-id>
+kradle experiment recordings <name> --all
+kradle experiment recordings <name> --version 2
+kradle experiment recordings <name> <run-id> --all
+```
+
+**Arguments:**
+| Argument | Description | Required |
+|----------|-------------|----------|
+| `name` | Experiment name | Yes |
+| `run-id` | Specific run ID to download | No |
+
+**Flags:**
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--all` | Download all runs/participants (skips interactive selection) | false |
+| `--version` | Specific experiment version number | Latest version |
+
+**Interactive mode (no flags):**
+1. Select a run from completed runs list
+2. Select participant(s) to download
+
+**Output location:**
+```
+experiments/<name>/versions/<version>/recordings/<run-id>/<participant-id>/
+```
+
+**Recording format:** `.mcpr` files (Minecraft replay format)
+
+**Examples:**
+```bash
+# Interactive selection
+kradle experiment recordings benchmark-v1
+
+# Download specific run (interactive participant selection)
+kradle experiment recordings benchmark-v1 abc123-run-id
+
+# Download all recordings from all runs
+kradle experiment recordings benchmark-v1 --all
+
+# Download from specific version
+kradle experiment recordings benchmark-v1 --version 1
+
+# Download all participants for a specific run
+kradle experiment recordings benchmark-v1 abc123-run-id --all
+
+# Combine version and all flags
+kradle experiment recordings benchmark-v1 --version 2 --all
+```
+
+---
+
+### `kradle experiment list`
+
+Lists all local experiments.
+
+**Usage:**
+```bash
+kradle experiment list
+```
+
+**Output format:**
+```
+Experiments:
+  - benchmark-v1 (3 versions)
+  - agent-comparison (1 version)
+  - difficulty-test (2 versions)
+```
+
+**Example:**
+```bash
+kradle experiment list
+```
+
+---
+
+## Agent Commands
+
+### `kradle agent list`
+
+Lists all agents registered in the Kradle system.
+
+**Usage:**
+```bash
+kradle agent list
+```
+
+**Output format:**
+```
+Agents:
+  - team-kradle:baseline-agent
+  - team-kradle:advanced-agent
+  - my-username:custom-agent
+```
+
+**Example:**
+```bash
+kradle agent list
+```
+
+---
+
+## AI Docs Commands
+
+These commands output LLM-focused documentation to stdout. Use them to get comprehensive reference material about the Kradle CLI and API.
+
+### `kradle ai-docs cli`
+
+Outputs the CLI reference documentation (this document).
+
+**Usage:**
+```bash
+kradle ai-docs cli
+```
+
+**Output:** The full CLI reference in Markdown format.
+
+**Example:**
+```bash
+# Pipe to an LLM or save to a file
+kradle ai-docs cli > cli-reference.md
+```
+
+---
+
+### `kradle ai-docs api`
+
+Outputs the API reference documentation for the `@kradle/challenges` package.
+
+**Usage:**
+```bash
+kradle ai-docs api
+```
+
+**Output:** The full API reference in Markdown format, including:
+- `createChallenge` API
+- Variables system
+- Events system
+- Actions library
+- Sandstone integration
+- Complete examples
+
+**Example:**
+```bash
+# Pipe to an LLM or save to a file
+kradle ai-docs api > api-reference.md
+```
+
+---
+
+## Common Workflows
+
+### Creating and Testing a New Challenge
+
+```bash
+# 1. Create the challenge
+kradle challenge create my-new-challenge
+
+# 2. Edit the challenge files
+# - challenges/my-new-challenge/challenge.ts (behavior)
+# - challenges/my-new-challenge/config.ts (metadata)
+
+# 3. Build and upload
+kradle challenge build my-new-challenge
+
+# 4. Test with watch mode during development
+kradle challenge watch my-new-challenge
+
+# 5. Run a test game
+kradle challenge run my-new-challenge
+```
+
+### Running a Benchmark Experiment
+
+```bash
+# 1. Create experiment
+kradle experiment create agent-benchmark
+
+# 2. Configure runs in experiments/agent-benchmark/config.ts
+# Define which agents to test, how many runs, etc.
+
+# 3. Run the experiment
+kradle experiment run agent-benchmark --max-concurrent 10
+
+# 4. If interrupted, resume with same command
+kradle experiment run agent-benchmark
+
+# 5. Download recordings for analysis
+kradle experiment recordings agent-benchmark --all
+```
+
+### Making a Challenge Public
+
+```bash
+# Build with public visibility
+kradle challenge build my-challenge --visibility public
+
+# Or build all challenges as public
+kradle challenge build --all --visibility public
+```
+
+---
+
+## Error Handling
+
+### Common Errors and Solutions
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `API call failed: 401` | Invalid or missing API key | Check `KRADLE_API_KEY` in `.env` |
+| `Challenge not found` | Challenge doesn't exist locally or in cloud | Run `kradle challenge create <name>` first |
+| `Experiment does not exist` | Experiment not created | Run `kradle experiment create <name>` first |
+| `Config file not found` | Missing `config.ts` | Ensure challenge/experiment has config file |
+| `No completed runs found` | Trying to download recordings before runs finish | Wait for runs to complete |
+
+### Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | Success |
+| 1 | General error (check error message) |
+| 2 | Invalid arguments or flags |
+
+---
+
+## Quick Reference
+
+| Command | Description |
+|---------|-------------|
+| `kradle init` | Initialize new project |
+| `kradle challenge create <name>` | Create new challenge |
+| `kradle challenge build <name>` | Build and upload challenge |
+| `kradle challenge build --all` | Build all challenges |
+| `kradle challenge delete <name>` | Delete challenge |
+| `kradle challenge list` | List all challenges |
+| `kradle challenge watch <name>` | Watch and auto-rebuild |
+| `kradle challenge run <name>` | Run challenge |
+| `kradle experiment create <name>` | Create new experiment |
+| `kradle experiment run <name>` | Run/resume experiment |
+| `kradle experiment recordings <name>` | Download recordings |
+| `kradle experiment list` | List experiments |
+| `kradle agent list` | List available agents |
+| `kradle ai-docs cli` | Output CLI reference for LLMs |
+| `kradle ai-docs api` | Output API reference for LLMs |

package/static/project_template/AGENTS.md

@@ -0,0 +1,15 @@
+# API & CLI reference
+
+To access Kradle's API & CLI references, use the following commands:
+
+```bash
+# Get CLI reference (commands, flags, workflows)
+kradle ai-docs cli
+
+# Get API reference (@kradle/challenges package documentation)
+kradle ai-docs api
+```
+
+These will output comprehensive documentation for working with Kradle challenges and experiments.
+
+You should always read both. You should create challenges and experiments with the CLI if possible - avoid creating them from scratch.

package/static/project_template/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md