kradle 0.0.2 → 0.3.1

package/README.md

@@ -0,0 +1,422 @@
+<!-- This file is for human consumption. If you are an LLM or any AI Agent, make sure to read static/ai_docs/LLM_CLI_REFERENCE.md for an exhaustive explanation of this package. -->
+
+# Kradle CLI
+
+Kradle's CLI for managing Minecraft challenges, experiments, agents, and more!
+
+* [Installation](#installation)
+* [Autocomplete](#autocomplete)
+* [Configuration](#configuration)
+* [Challenge](#challenge-commands)
+* [Experiments](#experiment-commands)
+* [Worlds](#world-commands)
+* [Agents](#agent-commands)
+* [AI Docs](#ai-docs-commands)
+* [Publishing a New Version](#publishing-a-new-version)
+* [Development](#development)
+* [Architecture](#architecture)
+
+## Installation
+
+1. Install Kradle's CLI globally
+```
+npm i -g kradle
+```
+2. Initialize a new directory to store challenges and experiments
+```
+kradle init
+```
+3. Congrats 🎉 You can now create a new challenge or a new experiment:
+```
+kradle challenge create <challenge-name>
+kradle experiment create <experiment-name>
+```
+
+In addition, you can enable [autocomplete](#Autocomplete).
+
+## Autocomplete
+
+Kradle CLI supports shell autocomplete for faster command entry. After installation, enable autocomplete for your shell:
+
+```bash
+kradle autocomplete
+# Follow the instructions printed
+```
+
+The command will display instructions for your specific shell.
+
+After setup, you will be able to use Tab to autocomplete:
+```bash
+kradle challenge <TAB>        # Shows: build, create, list, run, upload, watch, etc.
+```
+
+## Configuration
+
+The CLI requires a `.env` file with your Kradle API key and environment settings.
+
+**For new projects:** Run `kradle init` which will prompt for your API key and create the `.env` automatically.
+
+**Manual setup:** Create a `.env` file in your project root:
+```bash
+KRADLE_API_KEY=your-api-key-here
+KRADLE_API_URL=https://dev.kradle.ai/api
+```
+
+Get your API key at: https://dev.kradle.ai/settings#api-keys
+
+## Challenge Commands
+
+### Create Challenge
+
+Create a new challenge locally and in the cloud:
+
+```bash
+kradle challenge create <challenge-name>
+```
+
+This creates a `challenges/<challenge-name>/` folder with:
+- `challenge.ts`: The entrypoint defining challenge behavior
+- `config.ts`: TypeScript file with challenge metadata (auto-generated from cloud API)
+
+### Build Challenge
+
+Build challenge datapack and upload both config and datapack:
+
+```bash
+kradle challenge build <challenge-name>
+```
+
+This command:
+1. Creates the challenge in the cloud (if it doesn't already exists)
+2. Uploads `config.ts` to cloud (if it exists)
+3. Builds the datapack by executing `challenge.ts`
+4. Uploads the datapack to GCS
+
+### Delete Challenge
+
+Delete a challenge locally, from the cloud, or both:
+
+```bash
+# Will ask confirmation for local & cloud deletion
+kradle challenge delete <challenge-name>
+
+# Doesn't ask for confirmation
+kradle challenge delete <challenge-name> --yes
+```
+
+### List Challenges
+
+List all challenges (local and cloud):
+
+```bash
+kradle challenge list
+```
+
+### Pull Challenge
+
+Download a challenge from the cloud and extract source files locally:
+
+```bash
+kradle challenge pull                                  # Interactive selection
+kradle challenge pull <challenge-name>                 # Pull your own challenge
+kradle challenge pull <team-name>:<challenge-name>     # Pull a public challenge from another team
+kradle challenge pull <challenge-name> --yes           # Skip confirmation when overwriting
+```
+
+This downloads the challenge tarball, extracts `challenge.ts` and `config.ts`, and builds the datapack locally.
+
+### Watch Challenge
+
+Watch a challenge for changes and auto-rebuild/upload:
+
+```bash
+kradle challenge watch <challenge-name>
+```
+
+Uses file watching with debouncing (300ms) and hash comparison to minimize unnecessary rebuilds.
+
+### Run Challenge
+
+Run a challenge in production or studio environment:
+
+```bash
+kradle challenge run <challenge-name>
+kradle challenge run <challenge-name> --studio  # Run in local studio environment
+kradle challenge run <team-name>:<challenge-name>  # Run a public challenge from another team
+```
+
+## Experiment Commands
+
+Experiments allow you to run batches of challenge runs with different agents and configurations, then analyze the results. This is useful for benchmarking agents, testing challenge difficulty, or gathering statistics across many runs.
+
+### Concepts
+
+**Experiment**: A named collection of run configurations defined in a `config.ts` file. Each experiment lives in `experiments/<name>/`.
+
+**Version**: A snapshot of an experiment execution. When you run an experiment, it creates a version containing:
+- A copy of the `config.ts` at that point in time
+- A `manifest.json` with the generated list of runs
+- A `progress.json` tracking the status of each run
+
+Versions are stored in `experiments/<name>/versions/001/`, `002/`, etc. This allows you to:
+- Resume an interrupted experiment from where it left off
+- Re-run the same experiment with `--new` to create a fresh version
+- Compare results across different versions
+
+### Create Experiment
+
+Create a new experiment with a template config file:
+
+```bash
+kradle experiment create <name>
+```
+
+This creates `experiments/<name>/config.ts` with a template that you can customize. The config exports a `main()` function that returns a manifest with:
+- `runs`: Array of run configurations (challenge + participants)
+- `tags`: Optional tags applied to all runs for filtering in analytics
+
+### Run Experiment
+
+Execute or resume an experiment:
+
+```bash
+kradle experiment run <name>                       # Resume current version or create first one
+kradle experiment run <name> --new-version         # Start a new version
+kradle experiment run <name> --max-concurrent 10   # Control parallelism (default: 5)
+kradle experiment run <name> --download-recordings # Auto-download recordings as runs complete
+```
+
+The run command:
+1. Creates a new version (or resumes the current one)
+2. Generates a manifest by executing `config.ts`
+3. Displays an interactive TUI showing run progress
+4. Saves progress periodically (allows resuming if interrupted)
+5. Opens Metabase dashboard with results when complete
+
+### Download Recordings
+
+Download gameplay recordings from completed experiment runs:
+
+```bash
+kradle experiment recordings <name>                  # Interactive selection of run and participant
+kradle experiment recordings <name> <run-id>         # Download specific run
+kradle experiment recordings <name> --all            # Download all runs and participants
+kradle experiment recordings <name> --version 2      # Download from specific version
+kradle experiment recordings <name> <run-id> --all   # Download all participants for a run
+```
+
+Recordings are saved to `experiments/<name>/versions/<version>/recordings/<run-id>/`.
+
+### List Experiments
+
+List all local experiments:
+
+```bash
+kradle experiment list
+```
+
+## World Commands
+
+Worlds are Minecraft world saves that can be used as starting points for challenges.
+
+### Import World
+
+Import a Minecraft world folder from your local filesystem:
+
+```bash
+kradle world import ~/minecraft/saves/MyWorld              # Auto-generate slug from folder name
+kradle world import ~/minecraft/saves/MyWorld --as my-world  # Specify custom slug
+```
+
+This validates the folder contains `level.dat`, packages it as a tarball, creates a `config.ts`, and uploads to the cloud.
+
+### Push World
+
+Upload world config and tarball to the cloud:
+
+```bash
+kradle world push my-world                     # Push single world
+kradle world push my-world another-world       # Push multiple worlds
+kradle world push --all                        # Push all local worlds
+kradle world push my-world --public            # Push and set visibility to public
+```
+
+### Pull World
+
+Download a world from the cloud:
+
+```bash
+kradle world pull                              # Interactive selection
+kradle world pull my-world                     # Pull specific world
+kradle world pull username:their-world         # Pull from another user
+kradle world pull my-world --yes               # Skip confirmation when overwriting
+```
+
+### List Worlds
+
+List all worlds (local and cloud):
+
+```bash
+kradle world list
+```
+
+Shows sync status for each world (synced, cloud only, or local only).
+
+### Delete World
+
+Delete a world locally, from the cloud, or both:
+
+```bash
+kradle world delete my-world          # Interactive confirmation
+kradle world delete my-world --yes    # Skip confirmation
+```
+
+## Agent Commands
+
+### List Agents
+
+List all agents registered in the system:
+
+```bash
+kradle agent list
+```
+
+## AI Docs Commands
+
+Output LLM-focused documentation to stdout. These commands are designed to provide AI agents with comprehensive reference material about the Kradle CLI and API.
+
+### CLI Reference
+
+Output the CLI reference documentation:
+
+```bash
+kradle ai-docs cli
+```
+
+### Challenges SDK Reference
+
+Output the API reference documentation for the `@kradle/challenges-sdk` package:
+
+```bash
+kradle ai-docs challenges-sdk              # Uses locally installed or latest version
+kradle ai-docs challenges-sdk 0.2.1        # Uses specific version
+```
+
+This fetches the documentation from unpkg.com, matching the SDK version in your project.
+
+## Publishing a New Version
+
+The CLI uses GitHub Actions for automated releases. To publish a new version:
+
+1. **Go to Actions** in the GitHub repository
+2. **Select "Create Release PR"** workflow from the sidebar
+3. **Click "Run workflow"** and choose the release type:
+   - `patch` - Bug fixes (0.0.5 → 0.0.6)
+   - `minor` - New features (0.0.5 → 0.1.0)
+   - `major` - Breaking changes (0.0.5 → 1.0.0)
+4. **Review and merge** the automatically created PR
+5. **Done!** The package is automatically published to npm when the PR is merged
+
+
+## Development
+
+### Setup
+
+This CLI requires linking to be used locally:
+
+```bash
+npm install
+npm run build
+npm link
+```
+
+The repository provides the `kradle` CLI command. It runs compiled JavaScript from `dist/`:
+  - It requires running `npm run build` after every code change
+  - You can use `npm run watch` to make sure your code automatically recompiles after any change
+
+
+### Build & Lint
+
+```bash
+npm run build              # Compile TypeScript to dist/
+npm run lint               # Check for linting issues
+npm run lint:fix           # Auto-fix linting issues
+npm run format             # Format code with Biome
+```
+
+### Running Tests
+
+The CLI has integration tests that verify commands work correctly with the dev API.
+
+**Setup:**
+
+1. Copy `.env.test.example` to `.env.test`
+2. Add your Kradle API key (from https://dev.kradle.ai/settings/api-keys)
+
+```bash
+cp .env.test.example .env.test
+# Edit .env.test and add your API key
+```
+
+**Run tests:**
+
+```bash
+npm test                   # Run all tests
+npm run test:watch         # Run tests in watch mode
+npm run test:integration   # Run integration tests
+```
+
+**Note:** Integration tests make real API calls to the dev environment and may create/delete challenges.
+
+**CI Configuration:** Integration tests run in GitHub Actions on PRs. The `KRADLE_API_KEY` secret must be configured in the repository settings.
+
+### Challenge Structure
+
+Each challenge is a folder in `challenges/<slug>/` containing:
+
+- **`challenge.ts`**: Entrypoint that defines challenge behavior using the Sandstone API
+- **`config.ts`**: TypeScript file exporting challenge metadata (name, visibility, roles, objectives, etc.)
+
+**Workflow:**
+1. `kradle challenge create <slug>` creates the folder with `challenge.ts`
+2. The create command automatically builds, uploads, and downloads the config from the cloud API
+3. The downloaded JSON is converted into a typed TypeScript `config.ts` file
+4. `kradle challenge build <slug>` automatically uploads `config.ts` (if it exists) before building the datapack
+5. You can modify `config.ts` locally and run `build` to sync changes to the cloud
+
+## Architecture
+
+The CLI is built with:
+
+- **oclif**: CLI framework
+- **enquirer**: Interactive prompts
+- **listr2**: Task list UI
+- **ink**: React-based terminal UI (for experiments)
+- **react**: UI components for ink
+- **picocolors**: Terminal colors
+- **zod**: Schema validation
+- **chokidar**: File watching
+- **biome**: Linting and formatting
+
+### Project Structure
+
+```
+kradle-cli/
+├── src/
+│   ├── commands/             # CLI commands
+│   │   ├── agent/            # Agent commands
+│   │   ├── ai-docs/          # AI documentation commands
+│   │   ├── challenge/        # Challenge management commands
+│   │   ├── experiment/       # Experiment commands
+│   │   └── world/            # World management commands
+│   └── lib/                  # Core libraries
+│       └── experiment/       # Experiment system
+├── tests/                    # Integration tests
+│   ├── helpers/              # Test utilities
+│   └── integration/          # Integration test suites
+│       ├── challenge/        # Challenge command tests
+│       ├── experiment/       # Experiment command tests
+│       └── world/            # World command tests
+└── static/                   # Template files
+    └── project_template/     # Files for kradle init
+```

package/bin/run.cmd

@@ -0,0 +1,3 @@
+@echo off
+
+node "%~dp0\run" %*

package/bin/run.js

@@ -0,0 +1,14 @@
+#!/usr/bin/env -S node --no-warnings
+
+import { execute } from "@oclif/core";
+import path from "node:path";
+import fs from "node:fs";
+import dotenv from "dotenv";
+
+// Load .env file from cwd if it exists
+const envPath = path.join(process.cwd(), ".env");
+if (fs.existsSync(envPath)) {
+	dotenv.config({ path: envPath, quiet: true });
+}
+
+await execute({ dir: import.meta.url });

package/dist/commands/agent/list.d.ts

@@ -0,0 +1,10 @@
+import { Command } from "@oclif/core";
+export default class List extends Command {
+    static description: string;
+    static examples: string[];
+    static flags: {
+        "api-key": import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
+        "api-url": import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
+    };
+    run(): Promise<void>;
+}

package/dist/commands/agent/list.js

@@ -0,0 +1,22 @@
+import { Command } from "@oclif/core";
+import pc from "picocolors";
+import { ApiClient } from "../../lib/api-client.js";
+import { getConfigFlags } from "../../lib/flags.js";
+export default class List extends Command {
+    static description = "List all agents";
+    static examples = ["<%= config.bin %> <%= command.id %>"];
+    static flags = {
+        ...getConfigFlags("api-key", "api-url"),
+    };
+    async run() {
+        const { flags } = await this.parse(List);
+        const api = new ApiClient(flags["api-url"], flags["api-key"]);
+        this.log(pc.blue(">> Loading agents..."));
+        const agents = await api.listKradleAgents();
+        agents.sort((a, b) => a.username?.localeCompare(b.username || "") || 0);
+        this.log(pc.bold(`\nFound ${agents.length} agents:\n`));
+        for (const agent of agents) {
+            this.log(pc.bold(`- ${agent.username}`));
+        }
+    }
+}

package/dist/commands/ai-docs/challenges-sdk.d.ts

@@ -0,0 +1,21 @@
+import { Command } from "@oclif/core";
+export default class ChallengesSdk extends Command {
+    static args: {
+        version: import("@oclif/core/interfaces").Arg<string | undefined, Record<string, unknown>>;
+    };
+    static description: string;
+    static examples: string[];
+    run(): Promise<void>;
+    private fetchLLMReadme;
+    /**
+     * Get the LLM README from the installed version of the SDK.
+     * @returns The LLM README, or null if not found.
+     */
+    private getLLMReadmeFromInstalledVersion;
+    /**
+     * Detect the specified version of the SDK by walking up the directory tree
+     * looking for a package.json that contains the SDK dependency.
+     * @returns The specified version of the SDK, or null if not found.
+     */
+    private detectPackageJsonVersion;
+}

package/dist/commands/ai-docs/challenges-sdk.js

@@ -0,0 +1,100 @@
+import fs from "node:fs";
+import { createRequire } from "node:module";
+import path from "node:path";
+import { Args, Command } from "@oclif/core";
+import pc from "picocolors";
+const SDK_PACKAGE_NAME = "@kradle/challenges-sdk";
+const DOCS_FILENAME = "LLM_README.md";
+export default class ChallengesSdk extends Command {
+    static args = {
+        version: Args.string({
+            description: "SDK version to fetch docs for. If not specified, the locally installed version will be used if available, otherwise the latest version will be used.",
+            required: false,
+        }),
+    };
+    static description = "Output the @kradle/challenges-sdk API reference documentation for LLMs";
+    static examples = [
+        "<%= config.bin %> <%= command.id %>",
+        "<%= config.bin %> <%= command.id %> 0.2.1",
+        "<%= config.bin %> <%= command.id %> latest",
+    ];
+    async run() {
+        const { args } = await this.parse(ChallengesSdk);
+        if (args.version) {
+            const content = await this.fetchLLMReadme(args.version);
+            this.log(content);
+            return;
+        }
+        const llmReadmeFromInstalled = this.getLLMReadmeFromInstalledVersion();
+        if (llmReadmeFromInstalled) {
+            this.log(llmReadmeFromInstalled);
+            return;
+        }
+        this.log(pc.yellow("No version found. Trying to resolve version from package.json..."));
+        const packageJsonVersion = this.detectPackageJsonVersion();
+        if (packageJsonVersion) {
+            this.log(pc.yellow(`Using version ${packageJsonVersion} from package.json`));
+            const content = await this.fetchLLMReadme(packageJsonVersion);
+            this.log(content);
+            return;
+        }
+        this.log(pc.yellow("No version found. Using latest version."));
+        const content = await this.fetchLLMReadme("latest");
+        this.log(content);
+        return;
+    }
+    async fetchLLMReadme(version) {
+        const url = `https://cdn.jsdelivr.net/npm/${SDK_PACKAGE_NAME}@${version}/${DOCS_FILENAME}`;
+        try {
+            const response = await fetch(url);
+            return response.text();
+        }
+        catch (error) {
+            this.error("Network error: Could not reach jsdelivr.net. Check your internet connection.");
+            throw error;
+        }
+    }
+    /**
+     * Get the LLM README from the installed version of the SDK.
+     * @returns The LLM README, or null if not found.
+     */
+    getLLMReadmeFromInstalledVersion() {
+        try {
+            const require = createRequire(import.meta.url);
+            const sdkPkgPath = require.resolve(`${SDK_PACKAGE_NAME}/package.json`, {
+                paths: [process.cwd()],
+            });
+            const llmReadme = fs.readFileSync(path.join(path.dirname(sdkPkgPath), DOCS_FILENAME), "utf-8");
+            return llmReadme;
+        }
+        catch {
+            return null;
+        }
+    }
+    /**
+     * Detect the specified version of the SDK by walking up the directory tree
+     * looking for a package.json that contains the SDK dependency.
+     * @returns The specified version of the SDK, or null if not found.
+     */
+    detectPackageJsonVersion() {
+        let currentDir = process.cwd();
+        const root = path.parse(currentDir).root;
+        while (currentDir !== root) {
+            console.log("Checking", currentDir);
+            try {
+                const pkgPath = path.join(currentDir, "package.json");
+                const pkg = JSON.parse(fs.readFileSync(pkgPath, "utf-8"));
+                const deps = { ...pkg.dependencies, ...pkg.devDependencies };
+                const version = deps[SDK_PACKAGE_NAME];
+                if (version) {
+                    return version;
+                }
+            }
+            catch {
+                // package.json not found or invalid, continue to parent
+            }
+            currentDir = path.dirname(currentDir);
+        }
+        return null;
+    }
+}

package/dist/commands/ai-docs/cli.d.ts

@@ -0,0 +1,6 @@
+import { Command } from "@oclif/core";
+export default class Cli extends Command {
+    static description: string;
+    static examples: string[];
+    run(): Promise<void>;
+}

package/dist/commands/ai-docs/cli.js

@@ -0,0 +1,13 @@
+import fs from "node:fs/promises";
+import { Command } from "@oclif/core";
+import { getStaticResourcePath } from "../../lib/utils.js";
+export default class Cli extends Command {
+    static description = "Output the Kradle CLI reference documentation for LLMs";
+    static examples = ["<%= config.bin %> <%= command.id %>"];
+    async run() {
+        await this.parse(Cli);
+        const docPath = getStaticResourcePath("ai_docs/LLM_CLI_REFERENCE.md");
+        const content = await fs.readFile(docPath, "utf-8");
+        this.log(content);
+    }
+}

package/dist/commands/challenge/build.d.ts

@@ -0,0 +1,17 @@
+import { Command } from "@oclif/core";
+export default class Build extends Command {
+    static description: string;
+    static examples: string[];
+    static args: {
+        challengeSlug: import("@oclif/core/interfaces").Arg<string | undefined>;
+    };
+    static flags: {
+        "api-key": import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
+        "api-url": import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
+        "challenges-path": import("@oclif/core/interfaces").OptionFlag<string, import("@oclif/core/interfaces").CustomOptions>;
+        all: import("@oclif/core/interfaces").BooleanFlag<boolean>;
+        public: import("@oclif/core/interfaces").BooleanFlag<boolean>;
+    };
+    static strict: boolean;
+    run(): Promise<void>;
+}

package/dist/commands/challenge/build.js

@@ -0,0 +1,53 @@
+import { Command, Flags, loadHelpClass } from "@oclif/core";
+import pc from "picocolors";
+import { ApiClient } from "../../lib/api-client.js";
+import { getChallengeSlugArgument } from "../../lib/arguments.js";
+import { Challenge } from "../../lib/challenge.js";
+import { getConfigFlags } from "../../lib/flags.js";
+export default class Build extends Command {
+    static description = "Build and upload challenge datapack and config";
+    static examples = [
+        "<%= config.bin %> <%= command.id %> my-challenge",
+        "<%= config.bin %> <%= command.id %> my-challenge my-other-challenge",
+        "<%= config.bin %> <%= command.id %> --all",
+    ];
+    static args = {
+        challengeSlug: getChallengeSlugArgument({
+            description: "Challenge slug to build and upload. Can be used multiple times to build and upload multiple challenges at once. Incompatible with --all flag.",
+            required: false,
+        }),
+    };
+    static flags = {
+        all: Flags.boolean({ char: "a", description: "Build all challenges in the challenges directory", default: false }),
+        public: Flags.boolean({ char: "p", description: "Upload challenges as public.", default: false }),
+        ...getConfigFlags("api-key", "api-url", "challenges-path"),
+    };
+    static strict = false;
+    async run() {
+        const { argv, flags } = await this.parse(Build);
+        if (flags.all && argv.length > 0) {
+            this.error(pc.red("Cannot use --all flag with challenge slugs"));
+        }
+        if (!flags.all && argv.length === 0) {
+            // Show help if no challenge slugs are provided - https://github.com/oclif/oclif/issues/183#issuecomment-1933104981
+            await new (await loadHelpClass(this.config))(this.config).showHelp([Build.id]);
+            return;
+        }
+        if (flags.all) {
+            this.log(pc.blue("Building all challenges"));
+        }
+        const challengeSlugs = flags.all ? await Challenge.getLocalChallenges() : argv;
+        const api = new ApiClient(flags["api-url"], flags["api-key"]);
+        for (const challengeSlug of challengeSlugs) {
+            const challenge = new Challenge(challengeSlug, flags["challenges-path"]);
+            this.log(pc.blue(`==== Building challenge: ${challenge.shortSlug} ====`));
+            try {
+                await challenge.buildAndUpload(api, flags.public);
+            }
+            catch (error) {
+                this.error(pc.red(`Build failed: ${error instanceof Error ? error.message : String(error)}`));
+            }
+            this.log();
+        }
+    }
+}