@aigne/cli 1.74.0-beta → 1.74.0-beta.2

package/README.md

@@ -32,6 +32,7 @@ Command-line tool for [AIGNE Framework](https://github.com/ArcBlock/aigne-framew
 
 * **Project Creation**: Quickly create new AIGNE projects with predefined file structures and configurations
 * **Agent Running**: Easily run and test AIGNE agents
+* **Interactive Shell**: Full-featured REPL for LLM conversations with agent loading, session management, and built-in tools
 * **Testing Support**: Built-in test command for unit testing and integration testing
 * **MCP Services**: Support for launching agents as MCP servers for integration with external systems
 * **Interactive Interface**: Beautiful command-line interface providing an intuitive user experience
@@ -68,17 +69,32 @@ aigne --help
 # Create a new project
 aigne create [path]
 
+# Start interactive shell
+aigne shell
+
 # Run an agent
-aigne run --path xxx
+aigne run [path]
+
+# Run agent skills
+aigne skill <paths..>
 
 # Run tests
-aigne test --path xxx
+aigne test
+
+# Evaluate an agent
+aigne eval [path]
 
 # Start MCP server
-aigne serve-mcp --path xxx
+aigne serve-mcp [path]
 
 # Start observability server
-aigne observe [option]
+aigne observe
+
+# Manage AIGNE Hub connections
+aigne hub <command>
+
+# Get command documentation
+aigne explain [command]
 ```
 
 ## Create Command
@@ -98,30 +114,118 @@ The interactive creation process will ask for:
 * Project name
 * Project template (currently supports the default template)
 
+## Shell Command
+
+Start an interactive shell for LLM conversations with full agent support.
+
+```bash
+# Start shell with default model
+aigne shell
+
+# Start shell with specific model
+aigne shell --model gpt-4
+```
+
+### Shell Features
+
+* **Direct Mode**: Chat directly with LLM without loading agents
+* **Agent Mode**: Load and interact with AIGNE agents
+* **Conversation Context**: Maintains conversation history within session
+* **Built-in Tools**: `ask_user` for user input, `render` for rich output
+
+### Slash Commands
+
+| Command | Description |
+|---------|-------------|
+| `/help` | Show available commands |
+| `/exit`, `/quit` | Exit the shell |
+| `/load <path>` | Load an agent from a path |
+| `/list` | List loaded agents |
+| `/switch <name>` | Switch to a different agent |
+| `/unload <name>` | Unload an agent |
+| `/info` | Show current agent info |
+| `/mode [direct\|agent]` | Show or switch mode |
+| `/run <path>` | Run an agent (single execution) |
+| `/clear` | Clear conversation history |
+| `/model` | Show current model info |
+| `/explain <cmd>` | Explain CLI command |
+
+### Example Session
+
+```bash
+$ aigne shell
+aigne> Hello, what can you do?
+I'm a helpful AI assistant...
+
+aigne> /load ./my-agent
+Loaded: my-agent
+
+aigne [my-agent]> Process this data
+Processing...
+
+aigne [my-agent]> /mode direct
+Switched to direct mode.
+
+aigne> /exit
+Goodbye!
+```
+
 ## Run Command
 
-Launch a chat loop with the specified agent.
+Launch an agent or start an interactive chat session.
 
 ```bash
 # Run the agent in the current directory
 aigne run
 
 # Run the agent at the specified path
-aigne run --path path/to/agents
+aigne run path/to/agents
 
 # Run the agent from a remote URL
-aigne run --url https://example.com/aigne-project
+aigne run https://example.com/aigne-project
 
 # Run a specific agent
-aigne run --entry-agent myAgent
+aigne run path/to/agents myAgent
+
+# Run with input message
+aigne run -m "Hello, agent!"
 ```
 
 Available options:
 
-* `--entry-agent <entry-agent>` - Specify the agent name to run (defaults to the first agent found)
-* `--cache-dir <dir>` - Specify the directory to download the package to (used in URL mode)
+* `[entry-agent]` - Specify the agent name to run (defaults to the first agent found)
 * `--model <provider[:model]>` - Specify the AI model in format 'provider\[:model]' where model is optional (e.g., 'openai' or 'openai:gpt-4o-mini')
-* `--verbose` - Enable verbose logging
+* `-m, --input <message>` - Input message to send to the agent
+* `-f, --input-file <file>` - Input file to send to the agent
+* `--interactive, --chat` - Run in interactive chat mode
+* `--json` - Output in JSON format
+* `--view <type>` - Output view type (default|json|llm|human)
+* `--log-level <level>` - Log level (silent|error|warn|info|debug)
+
+## Skill Command
+
+Run Agent Skills from specified directories.
+
+```bash
+# Run a single skill
+aigne skill ./my-skill
+
+# Run multiple skills
+aigne skill ./skill1 ./skill2
+
+# Run skill with input
+aigne skill ./skill -m "Process this data"
+
+# Run skill in interactive mode
+aigne skill ./skill --interactive
+```
+
+Available options:
+
+* `<paths..>` - One or more paths to Agent Skill directories (required)
+* `-m, --input <message>` - Input message for the skill
+* `--interactive, --chat` - Run in interactive chat mode
+* `--model <provider[:model]>` - Specify the AI model
 
 ## Test Command
 
@@ -150,18 +254,53 @@ aigne serve-mcp --port 3001
 aigne serve-mcp --path path/to/agents
 ```
 
-## Serve Command (observability)
+## Observe Command
 
-Start the service for monitoring data
+Start the observability server for monitoring and debugging agents.
 
 ```bash
-# Start observability server on default port 7890
+# Start observability server on default port
 aigne observe
 
 # Start observability server on specified port
 aigne observe --port 3001
 ```
 
+## Hub Command
+
+Manage AIGNE Hub connections for accessing AI models.
+
+```bash
+# List all connected hubs
+aigne hub list
+
+# Connect to an AIGNE Hub
+aigne hub connect
+
+# Show connection status
+aigne hub status
+
+# Switch active hub
+aigne hub use
+
+# Remove a hub connection
+aigne hub remove
+```
+
+## Explain Command
+
+Get machine-readable documentation for commands (useful for LLMs and agents).
+
+```bash
+# List all available commands
+aigne explain
+
+# Get documentation for a specific command
+aigne explain run
+aigne explain skill
+aigne explain hub list
+```
+
 ## License
 
 Elastic-2.0

package/dist/commands/aigne.cjs

@@ -1,22 +1,21 @@
 const require_rolldown_runtime = require('../_virtual/rolldown_runtime.cjs');
 const require_constants = require('../constants.cjs');
 const require_utils_ascii_logo = require('../utils/ascii-logo.cjs');
-const require_app = require('./app.cjs');
 const require_create = require('./create.cjs');
-const require_deploy = require('./deploy.cjs');
 const require_eval = require('./eval.cjs');
+const require_explain = require('./explain.cjs');
 const require_hub = require('./hub.cjs');
 const require_observe = require('./observe.cjs');
 const require_serve_mcp = require('./serve-mcp.cjs');
 const require_run = require('./run.cjs');
-const require_run_skill = require('./run-skill.cjs');
+const require_shell = require('./shell.cjs');
 const require_test = require('./test.cjs');
 let yargs = require("yargs");
 yargs = require_rolldown_runtime.__toESM(yargs);
 
 //#region src/commands/aigne.ts
 function createAIGNECommand(options) {
-	return (0, yargs.default)().scriptName("aigne").usage(`${require_utils_ascii_logo.asciiLogo}\n$0 <command> [options]`).version(require_constants.AIGNE_CLI_VERSION).command(require_run.createRunCommand(options)).command(require_run_skill.createRunSkillCommand()).command(require_eval.createEvalCommand(options)).command(require_test.createTestCommand(options)).command(require_create.createCreateCommand()).command(require_serve_mcp.createServeMCPCommand(options)).command(require_observe.createObservabilityCommand()).command(require_app.createAppCommands(options)).command(require_hub.createHubCommand()).command(require_deploy.createDeployCommands()).demandCommand().version(false).alias("help", "h").wrap(null).strict();
+	return (0, yargs.default)().scriptName("aigne").usage(`${require_utils_ascii_logo.asciiLogo}\n$0 <command> [options]`).version(require_constants.AIGNE_CLI_VERSION).command(require_run.createRunCommand(options)).command(require_eval.createEvalCommand(options)).command(require_test.createTestCommand(options)).command(require_create.createCreateCommand()).command(require_serve_mcp.createServeMCPCommand(options)).command(require_observe.createObservabilityCommand()).command(require_hub.createHubCommand()).command(require_explain.createExplainCommand()).command(require_shell.createShellCommand()).demandCommand().version(false).alias("help", "h").wrap(null).strict();
 }
 
 //#endregion

package/dist/commands/aigne.mjs

@@ -1,20 +1,19 @@
 import { AIGNE_CLI_VERSION } from "../constants.mjs";
 import { asciiLogo } from "../utils/ascii-logo.mjs";
-import { createAppCommands } from "./app.mjs";
 import { createCreateCommand } from "./create.mjs";
-import { createDeployCommands } from "./deploy.mjs";
 import { createEvalCommand } from "./eval.mjs";
+import { createExplainCommand } from "./explain.mjs";
 import { createHubCommand } from "./hub.mjs";
 import { createObservabilityCommand } from "./observe.mjs";
 import { createServeMCPCommand } from "./serve-mcp.mjs";
 import { createRunCommand } from "./run.mjs";
-import { createRunSkillCommand } from "./run-skill.mjs";
+import { createShellCommand } from "./shell.mjs";
 import { createTestCommand } from "./test.mjs";
 import yargs from "yargs";
 
 //#region src/commands/aigne.ts
 function createAIGNECommand(options) {
-	return yargs().scriptName("aigne").usage(`${asciiLogo}\n$0 <command> [options]`).version(AIGNE_CLI_VERSION).command(createRunCommand(options)).command(createRunSkillCommand()).command(createEvalCommand(options)).command(createTestCommand(options)).command(createCreateCommand()).command(createServeMCPCommand(options)).command(createObservabilityCommand()).command(createAppCommands(options)).command(createHubCommand()).command(createDeployCommands()).demandCommand().version(false).alias("help", "h").wrap(null).strict();
+	return yargs().scriptName("aigne").usage(`${asciiLogo}\n$0 <command> [options]`).version(AIGNE_CLI_VERSION).command(createRunCommand(options)).command(createEvalCommand(options)).command(createTestCommand(options)).command(createCreateCommand()).command(createServeMCPCommand(options)).command(createObservabilityCommand()).command(createHubCommand()).command(createExplainCommand()).command(createShellCommand()).demandCommand().version(false).alias("help", "h").wrap(null).strict();
 }
 
 //#endregion

package/dist/commands/aigne.mjs.map

@@ -1 +1 @@
-{"version":3,"file":"aigne.mjs","names":[],"sources":["../../src/commands/aigne.ts"],"sourcesContent":["import yargs from \"yargs\";\nimport { AIGNE_CLI_VERSION } from \"../constants.js\";\nimport { asciiLogo } from \"../utils/ascii-logo.js\";\nimport { createAppCommands } from \"./app.js\";\nimport { createCreateCommand } from \"./create.js\";\nimport { createDeployCommands } from \"./deploy.js\";\nimport { createEvalCommand } from \"./eval.js\";\nimport { createHubCommand } from \"./hub.js\";\nimport { createObservabilityCommand } from \"./observe.js\";\nimport { createRunCommand } from \"./run.js\";\nimport { createRunSkillCommand } from \"./run-skill.js\";\nimport { createServeMCPCommand } from \"./serve-mcp.js\";\nimport { createTestCommand } from \"./test.js\";\n\nexport function createAIGNECommand(options?: { argv?: string[]; aigneFilePath?: string }) {\n  return (\n    yargs()\n      .scriptName(\"aigne\")\n      .usage(`${asciiLogo}\\n$0 <command> [options]`)\n      .version(AIGNE_CLI_VERSION)\n      // default command: when user runs `aigne` without subcommand, behave like `aigne run`\n      .command(createRunCommand(options))\n      .command(createRunSkillCommand())\n      .command(createEvalCommand(options))\n      .command(createTestCommand(options))\n      .command(createCreateCommand())\n      .command(createServeMCPCommand(options))\n      .command(createObservabilityCommand())\n      .command(createAppCommands(options))\n      .command(createHubCommand())\n      .command(createDeployCommands())\n      .demandCommand()\n      .version(false)\n      .alias(\"help\", \"h\")\n      .wrap(null)\n      .strict()\n  );\n}\n"],"mappings":";;;;;;;;;;;;;;;AAcA,SAAgB,mBAAmB,SAAuD;AACxF,QACE,OAAO,CACJ,WAAW,QAAQ,CACnB,MAAM,GAAG,UAAU,0BAA0B,CAC7C,QAAQ,kBAAkB,CAE1B,QAAQ,iBAAiB,QAAQ,CAAC,CAClC,QAAQ,uBAAuB,CAAC,CAChC,QAAQ,kBAAkB,QAAQ,CAAC,CACnC,QAAQ,kBAAkB,QAAQ,CAAC,CACnC,QAAQ,qBAAqB,CAAC,CAC9B,QAAQ,sBAAsB,QAAQ,CAAC,CACvC,QAAQ,4BAA4B,CAAC,CACrC,QAAQ,kBAAkB,QAAQ,CAAC,CACnC,QAAQ,kBAAkB,CAAC,CAC3B,QAAQ,sBAAsB,CAAC,CAC/B,eAAe,CACf,QAAQ,MAAM,CACd,MAAM,QAAQ,IAAI,CAClB,KAAK,KAAK,CACV,QAAQ"}
+{"version":3,"file":"aigne.mjs","names":[],"sources":["../../src/commands/aigne.ts"],"sourcesContent":["import yargs from \"yargs\";\nimport { AIGNE_CLI_VERSION } from \"../constants.js\";\nimport { asciiLogo } from \"../utils/ascii-logo.js\";\nimport { createCreateCommand } from \"./create.js\";\nimport { createEvalCommand } from \"./eval.js\";\nimport { createExplainCommand } from \"./explain.js\";\nimport { createHubCommand } from \"./hub.js\";\nimport { createObservabilityCommand } from \"./observe.js\";\nimport { createRunCommand } from \"./run.js\";\nimport { createServeMCPCommand } from \"./serve-mcp.js\";\nimport { createShellCommand } from \"./shell.js\";\nimport { createTestCommand } from \"./test.js\";\n\nexport function createAIGNECommand(options?: { argv?: string[]; aigneFilePath?: string }) {\n  return (\n    yargs()\n      .scriptName(\"aigne\")\n      .usage(`${asciiLogo}\\n$0 <command> [options]`)\n      .version(AIGNE_CLI_VERSION)\n      // default command: when user runs `aigne` without subcommand, behave like `aigne run`\n      .command(createRunCommand(options))\n      .command(createEvalCommand(options))\n      .command(createTestCommand(options))\n      .command(createCreateCommand())\n      .command(createServeMCPCommand(options))\n      .command(createObservabilityCommand())\n      .command(createHubCommand())\n      .command(createExplainCommand())\n      .command(createShellCommand())\n      .demandCommand()\n      .version(false)\n      .alias(\"help\", \"h\")\n      .wrap(null)\n      .strict()\n  );\n}\n"],"mappings":";;;;;;;;;;;;;;AAaA,SAAgB,mBAAmB,SAAuD;AACxF,QACE,OAAO,CACJ,WAAW,QAAQ,CACnB,MAAM,GAAG,UAAU,0BAA0B,CAC7C,QAAQ,kBAAkB,CAE1B,QAAQ,iBAAiB,QAAQ,CAAC,CAClC,QAAQ,kBAAkB,QAAQ,CAAC,CACnC,QAAQ,kBAAkB,QAAQ,CAAC,CACnC,QAAQ,qBAAqB,CAAC,CAC9B,QAAQ,sBAAsB,QAAQ,CAAC,CACvC,QAAQ,4BAA4B,CAAC,CACrC,QAAQ,kBAAkB,CAAC,CAC3B,QAAQ,sBAAsB,CAAC,CAC/B,QAAQ,oBAAoB,CAAC,CAC7B,eAAe,CACf,QAAQ,MAAM,CACd,MAAM,QAAQ,IAAI,CAClB,KAAK,KAAK,CACV,QAAQ"}

package/dist/commands/app/agent.cjs

@@ -2,37 +2,13 @@ const require_rolldown_runtime = require('../../_virtual/rolldown_runtime.cjs');
 const require_constants = require('../../constants.cjs');
 const require_utils_yargs = require('../../utils/yargs.cjs');
 const require_utils_run_with_aigne = require('../../utils/run-with-aigne.cjs');
-const require_serve_mcp = require('../serve-mcp.cjs');
+require('../serve-mcp.cjs');
 let _aigne_core_utils_logger = require("@aigne/core/utils/logger");
 let node_assert = require("node:assert");
 node_assert = require_rolldown_runtime.__toESM(node_assert);
 let _aigne_uuid = require("@aigne/uuid");
 
 //#region src/commands/app/agent.ts
-const serveMcpCommandModule = ({ aigne, name }) => ({
-	command: "serve-mcp",
-	describe: `Serve ${name} a MCP server (streamable http)`,
-	builder: (yargs) => {
-		return yargs.option("host", {
-			describe: "Host to run the MCP server on, use 0.0.0.0 to publicly expose the server",
-			type: "string",
-			default: "localhost"
-		}).option("port", {
-			describe: "Port to run the MCP server on",
-			type: "number"
-		}).option("pathname", {
-			describe: "Pathname to the service",
-			type: "string",
-			default: "/mcp"
-		});
-	},
-	handler: async (options) => {
-		await require_serve_mcp.serveMCPServerFromDir({
-			...options,
-			aigne
-		});
-	}
-});
 const agentCommandModule = ({ aigne, agent, interactive }) => {
 	return {
 		command: agent.name,
@@ -113,5 +89,4 @@ async function invokeAgent(options) {
 //#endregion
 exports.agentCommandModule = agentCommandModule;
 exports.cliAgentCommandModule = cliAgentCommandModule;
-exports.invokeAgent = invokeAgent;
-exports.serveMcpCommandModule = serveMcpCommandModule;
+exports.invokeAgent = invokeAgent;

package/dist/commands/app/agent.mjs

@@ -1,36 +1,12 @@
 import { DEFAULT_USER_ID } from "../../constants.mjs";
 import { parseAgentInput, withAgentInputSchema } from "../../utils/yargs.mjs";
 import { runAgentWithAIGNE } from "../../utils/run-with-aigne.mjs";
-import { serveMCPServerFromDir } from "../serve-mcp.mjs";
+import "../serve-mcp.mjs";
 import { logger } from "@aigne/core/utils/logger";
 import assert from "node:assert";
 import { v7 } from "@aigne/uuid";
 
 //#region src/commands/app/agent.ts
-const serveMcpCommandModule = ({ aigne, name }) => ({
-	command: "serve-mcp",
-	describe: `Serve ${name} a MCP server (streamable http)`,
-	builder: (yargs) => {
-		return yargs.option("host", {
-			describe: "Host to run the MCP server on, use 0.0.0.0 to publicly expose the server",
-			type: "string",
-			default: "localhost"
-		}).option("port", {
-			describe: "Port to run the MCP server on",
-			type: "number"
-		}).option("pathname", {
-			describe: "Pathname to the service",
-			type: "string",
-			default: "/mcp"
-		});
-	},
-	handler: async (options) => {
-		await serveMCPServerFromDir({
-			...options,
-			aigne
-		});
-	}
-});
 const agentCommandModule = ({ aigne, agent, interactive }) => {
 	return {
 		command: agent.name,
@@ -109,5 +85,5 @@ async function invokeAgent(options) {
 }
 
 //#endregion
-export { agentCommandModule, cliAgentCommandModule, invokeAgent, serveMcpCommandModule };
+export { agentCommandModule, cliAgentCommandModule, invokeAgent };
 //# sourceMappingURL=agent.mjs.map

package/dist/commands/app/agent.mjs.map

@@ -1 +1 @@
-{"version":3,"file":"agent.mjs","names":[],"sources":["../../../src/commands/app/agent.ts"],"sourcesContent":["import assert from \"node:assert\";\nimport { DEFAULT_USER_ID } from \"@aigne/cli/constants.js\";\nimport type { Agent, AIGNE, Message } from \"@aigne/core\";\nimport type { CLIAgent } from \"@aigne/core/utils/agent-utils\";\nimport { logger } from \"@aigne/core/utils/logger\";\nimport { v7 } from \"@aigne/uuid\";\nimport type { CommandModule } from \"yargs\";\nimport { runAgentWithAIGNE } from \"../../utils/run-with-aigne.js\";\nimport {\n  type AgentRunCommonOptions,\n  parseAgentInput,\n  withAgentInputSchema,\n} from \"../../utils/yargs.js\";\nimport { serveMCPServerFromDir } from \"../serve-mcp.js\";\n\nexport const serveMcpCommandModule = ({\n  aigne,\n  name,\n}: {\n  aigne: AIGNE;\n  name: string;\n}): CommandModule<unknown, { host: string; port?: number; pathname: string }> => ({\n  command: \"serve-mcp\",\n  describe: `Serve ${name} a MCP server (streamable http)`,\n  builder: (yargs) => {\n    return yargs\n      .option(\"host\", {\n        describe: \"Host to run the MCP server on, use 0.0.0.0 to publicly expose the server\",\n        type: \"string\",\n        default: \"localhost\",\n      })\n      .option(\"port\", {\n        describe: \"Port to run the MCP server on\",\n        type: \"number\",\n      })\n      .option(\"pathname\", {\n        describe: \"Pathname to the service\",\n        type: \"string\",\n        default: \"/mcp\",\n      });\n  },\n  handler: async (options) => {\n    await serveMCPServerFromDir({\n      ...options,\n      aigne,\n    });\n  },\n});\n\nexport const agentCommandModule = ({\n  aigne,\n  agent,\n  interactive,\n}: {\n  aigne: AIGNE;\n  agent: Agent;\n  interactive?: boolean;\n}): CommandModule<unknown, AgentRunCommonOptions> => {\n  return {\n    command: agent.name,\n    aliases: agent.alias || [],\n    describe: agent.description || \"\",\n    builder: async (yargs) => {\n      return withAgentInputSchema(yargs, {\n        inputSchema: agent.inputSchema,\n        optionalInputs:\n          interactive && \"inputKey\" in agent && typeof agent.inputKey === \"string\"\n            ? [agent.inputKey]\n            : [],\n      });\n    },\n    handler: async (options) => {\n      if (options.logLevel) logger.level = options.logLevel;\n\n      await invokeAgent({\n        aigne,\n        agent,\n        input: { ...options, interactive: interactive ?? options.interactive },\n      });\n    },\n  };\n};\n\nexport const cliAgentCommandModule = ({\n  aigne,\n  cliAgent,\n}: {\n  aigne: AIGNE;\n  cliAgent: CLIAgent<Agent>;\n}): CommandModule<unknown, AgentRunCommonOptions> => {\n  const { agent, agents } = cliAgent;\n\n  const name = cliAgent.name || agent?.name;\n  assert(name, \"CLI agent must have a name\");\n\n  return {\n    command: name,\n    aliases: cliAgent.alias || agent?.alias || [],\n    describe: cliAgent.description || agent?.description || \"\",\n    builder: async (yargs) => {\n      if (agent) {\n        withAgentInputSchema(yargs, { inputSchema: agent.inputSchema });\n      }\n      if (agents?.length) {\n        for (const cmd of agents) {\n          yargs.command(\n            cliAgentCommandModule({\n              aigne,\n              cliAgent: cmd,\n            }),\n          );\n        }\n      }\n\n      if (!agent) yargs.demandCommand();\n\n      return yargs;\n    },\n    handler: async (options) => {\n      if (!agent) throw new Error(\"CLI agent is not defined\");\n\n      if (options.logLevel) logger.level = options.logLevel;\n\n      await invokeAgent({\n        aigne,\n        agent,\n        input: options,\n      });\n    },\n  };\n};\n\nexport async function invokeAgent(options: {\n  aigne: AIGNE;\n  agent: Agent;\n  input: Message & AgentRunCommonOptions;\n}) {\n  const { agent, aigne } = options;\n\n  try {\n    const input = await parseAgentInput(options.input, agent);\n\n    const sessionId = options.input.sessionId || v7();\n    const userId = DEFAULT_USER_ID;\n\n    if (aigne.cli.init) {\n      await runAgentWithAIGNE(aigne, aigne.cli.init, {\n        ...options.input,\n        input,\n        interactive: false,\n        sessionId,\n        userId,\n      });\n    }\n\n    await runAgentWithAIGNE(aigne, agent, {\n      ...options.input,\n      input,\n      interactive: options.input.interactive,\n      sessionId,\n      userId,\n    });\n  } finally {\n    await aigne.shutdown();\n  }\n}\n"],"mappings":";;;;;;;;;AAeA,MAAa,yBAAyB,EACpC,OACA,YAIgF;CAChF,SAAS;CACT,UAAU,SAAS,KAAK;CACxB,UAAU,UAAU;AAClB,SAAO,MACJ,OAAO,QAAQ;GACd,UAAU;GACV,MAAM;GACN,SAAS;GACV,CAAC,CACD,OAAO,QAAQ;GACd,UAAU;GACV,MAAM;GACP,CAAC,CACD,OAAO,YAAY;GAClB,UAAU;GACV,MAAM;GACN,SAAS;GACV,CAAC;;CAEN,SAAS,OAAO,YAAY;AAC1B,QAAM,sBAAsB;GAC1B,GAAG;GACH;GACD,CAAC;;CAEL;AAED,MAAa,sBAAsB,EACjC,OACA,OACA,kBAKmD;AACnD,QAAO;EACL,SAAS,MAAM;EACf,SAAS,MAAM,SAAS,EAAE;EAC1B,UAAU,MAAM,eAAe;EAC/B,SAAS,OAAO,UAAU;AACxB,UAAO,qBAAqB,OAAO;IACjC,aAAa,MAAM;IACnB,gBACE,eAAe,cAAc,SAAS,OAAO,MAAM,aAAa,WAC5D,CAAC,MAAM,SAAS,GAChB,EAAE;IACT,CAAC;;EAEJ,SAAS,OAAO,YAAY;AAC1B,OAAI,QAAQ,SAAU,QAAO,QAAQ,QAAQ;AAE7C,SAAM,YAAY;IAChB;IACA;IACA,OAAO;KAAE,GAAG;KAAS,aAAa,eAAe,QAAQ;KAAa;IACvE,CAAC;;EAEL;;AAGH,MAAa,yBAAyB,EACpC,OACA,eAImD;CACnD,MAAM,EAAE,OAAO,WAAW;CAE1B,MAAM,OAAO,SAAS,QAAQ,OAAO;AACrC,QAAO,MAAM,6BAA6B;AAE1C,QAAO;EACL,SAAS;EACT,SAAS,SAAS,SAAS,OAAO,SAAS,EAAE;EAC7C,UAAU,SAAS,eAAe,OAAO,eAAe;EACxD,SAAS,OAAO,UAAU;AACxB,OAAI,MACF,sBAAqB,OAAO,EAAE,aAAa,MAAM,aAAa,CAAC;AAEjE,OAAI,QAAQ,OACV,MAAK,MAAM,OAAO,OAChB,OAAM,QACJ,sBAAsB;IACpB;IACA,UAAU;IACX,CAAC,CACH;AAIL,OAAI,CAAC,MAAO,OAAM,eAAe;AAEjC,UAAO;;EAET,SAAS,OAAO,YAAY;AAC1B,OAAI,CAAC,MAAO,OAAM,IAAI,MAAM,2BAA2B;AAEvD,OAAI,QAAQ,SAAU,QAAO,QAAQ,QAAQ;AAE7C,SAAM,YAAY;IAChB;IACA;IACA,OAAO;IACR,CAAC;;EAEL;;AAGH,eAAsB,YAAY,SAI/B;CACD,MAAM,EAAE,OAAO,UAAU;AAEzB,KAAI;EACF,MAAM,QAAQ,MAAM,gBAAgB,QAAQ,OAAO,MAAM;EAEzD,MAAM,YAAY,QAAQ,MAAM,aAAa,IAAI;EACjD,MAAM,SAAS;AAEf,MAAI,MAAM,IAAI,KACZ,OAAM,kBAAkB,OAAO,MAAM,IAAI,MAAM;GAC7C,GAAG,QAAQ;GACX;GACA,aAAa;GACb;GACA;GACD,CAAC;AAGJ,QAAM,kBAAkB,OAAO,OAAO;GACpC,GAAG,QAAQ;GACX;GACA,aAAa,QAAQ,MAAM;GAC3B;GACA;GACD,CAAC;WACM;AACR,QAAM,MAAM,UAAU"}
+{"version":3,"file":"agent.mjs","names":[],"sources":["../../../src/commands/app/agent.ts"],"sourcesContent":["import assert from \"node:assert\";\nimport { DEFAULT_USER_ID } from \"@aigne/cli/constants.js\";\nimport type { Agent, AIGNE, Message } from \"@aigne/core\";\nimport type { CLIAgent } from \"@aigne/core/utils/agent-utils\";\nimport { logger } from \"@aigne/core/utils/logger\";\nimport { v7 } from \"@aigne/uuid\";\nimport type { CommandModule } from \"yargs\";\nimport { runAgentWithAIGNE } from \"../../utils/run-with-aigne.js\";\nimport {\n  type AgentRunCommonOptions,\n  parseAgentInput,\n  withAgentInputSchema,\n} from \"../../utils/yargs.js\";\nimport { serveMCPServerFromDir } from \"../serve-mcp.js\";\n\nexport const serveMcpCommandModule = ({\n  aigne,\n  name,\n}: {\n  aigne: AIGNE;\n  name: string;\n}): CommandModule<unknown, { host: string; port?: number; pathname: string }> => ({\n  command: \"serve-mcp\",\n  describe: `Serve ${name} a MCP server (streamable http)`,\n  builder: (yargs) => {\n    return yargs\n      .option(\"host\", {\n        describe: \"Host to run the MCP server on, use 0.0.0.0 to publicly expose the server\",\n        type: \"string\",\n        default: \"localhost\",\n      })\n      .option(\"port\", {\n        describe: \"Port to run the MCP server on\",\n        type: \"number\",\n      })\n      .option(\"pathname\", {\n        describe: \"Pathname to the service\",\n        type: \"string\",\n        default: \"/mcp\",\n      });\n  },\n  handler: async (options) => {\n    await serveMCPServerFromDir({\n      ...options,\n      aigne,\n    });\n  },\n});\n\nexport const agentCommandModule = ({\n  aigne,\n  agent,\n  interactive,\n}: {\n  aigne: AIGNE;\n  agent: Agent;\n  interactive?: boolean;\n}): CommandModule<unknown, AgentRunCommonOptions> => {\n  return {\n    command: agent.name,\n    aliases: agent.alias || [],\n    describe: agent.description || \"\",\n    builder: async (yargs) => {\n      return withAgentInputSchema(yargs, {\n        inputSchema: agent.inputSchema,\n        optionalInputs:\n          interactive && \"inputKey\" in agent && typeof agent.inputKey === \"string\"\n            ? [agent.inputKey]\n            : [],\n      });\n    },\n    handler: async (options) => {\n      if (options.logLevel) logger.level = options.logLevel;\n\n      await invokeAgent({\n        aigne,\n        agent,\n        input: { ...options, interactive: interactive ?? options.interactive },\n      });\n    },\n  };\n};\n\nexport const cliAgentCommandModule = ({\n  aigne,\n  cliAgent,\n}: {\n  aigne: AIGNE;\n  cliAgent: CLIAgent<Agent>;\n}): CommandModule<unknown, AgentRunCommonOptions> => {\n  const { agent, agents } = cliAgent;\n\n  const name = cliAgent.name || agent?.name;\n  assert(name, \"CLI agent must have a name\");\n\n  return {\n    command: name,\n    aliases: cliAgent.alias || agent?.alias || [],\n    describe: cliAgent.description || agent?.description || \"\",\n    builder: async (yargs) => {\n      if (agent) {\n        withAgentInputSchema(yargs, { inputSchema: agent.inputSchema });\n      }\n      if (agents?.length) {\n        for (const cmd of agents) {\n          yargs.command(\n            cliAgentCommandModule({\n              aigne,\n              cliAgent: cmd,\n            }),\n          );\n        }\n      }\n\n      if (!agent) yargs.demandCommand();\n\n      return yargs;\n    },\n    handler: async (options) => {\n      if (!agent) throw new Error(\"CLI agent is not defined\");\n\n      if (options.logLevel) logger.level = options.logLevel;\n\n      await invokeAgent({\n        aigne,\n        agent,\n        input: options,\n      });\n    },\n  };\n};\n\nexport async function invokeAgent(options: {\n  aigne: AIGNE;\n  agent: Agent;\n  input: Message & AgentRunCommonOptions;\n}) {\n  const { agent, aigne } = options;\n\n  try {\n    const input = await parseAgentInput(options.input, agent);\n\n    const sessionId = options.input.sessionId || v7();\n    const userId = DEFAULT_USER_ID;\n\n    if (aigne.cli.init) {\n      await runAgentWithAIGNE(aigne, aigne.cli.init, {\n        ...options.input,\n        input,\n        interactive: false,\n        sessionId,\n        userId,\n      });\n    }\n\n    await runAgentWithAIGNE(aigne, agent, {\n      ...options.input,\n      input,\n      interactive: options.input.interactive,\n      sessionId,\n      userId,\n    });\n  } finally {\n    await aigne.shutdown();\n  }\n}\n"],"mappings":";;;;;;;;;AAiDA,MAAa,sBAAsB,EACjC,OACA,OACA,kBAKmD;AACnD,QAAO;EACL,SAAS,MAAM;EACf,SAAS,MAAM,SAAS,EAAE;EAC1B,UAAU,MAAM,eAAe;EAC/B,SAAS,OAAO,UAAU;AACxB,UAAO,qBAAqB,OAAO;IACjC,aAAa,MAAM;IACnB,gBACE,eAAe,cAAc,SAAS,OAAO,MAAM,aAAa,WAC5D,CAAC,MAAM,SAAS,GAChB,EAAE;IACT,CAAC;;EAEJ,SAAS,OAAO,YAAY;AAC1B,OAAI,QAAQ,SAAU,QAAO,QAAQ,QAAQ;AAE7C,SAAM,YAAY;IAChB;IACA;IACA,OAAO;KAAE,GAAG;KAAS,aAAa,eAAe,QAAQ;KAAa;IACvE,CAAC;;EAEL;;AAGH,MAAa,yBAAyB,EACpC,OACA,eAImD;CACnD,MAAM,EAAE,OAAO,WAAW;CAE1B,MAAM,OAAO,SAAS,QAAQ,OAAO;AACrC,QAAO,MAAM,6BAA6B;AAE1C,QAAO;EACL,SAAS;EACT,SAAS,SAAS,SAAS,OAAO,SAAS,EAAE;EAC7C,UAAU,SAAS,eAAe,OAAO,eAAe;EACxD,SAAS,OAAO,UAAU;AACxB,OAAI,MACF,sBAAqB,OAAO,EAAE,aAAa,MAAM,aAAa,CAAC;AAEjE,OAAI,QAAQ,OACV,MAAK,MAAM,OAAO,OAChB,OAAM,QACJ,sBAAsB;IACpB;IACA,UAAU;IACX,CAAC,CACH;AAIL,OAAI,CAAC,MAAO,OAAM,eAAe;AAEjC,UAAO;;EAET,SAAS,OAAO,YAAY;AAC1B,OAAI,CAAC,MAAO,OAAM,IAAI,MAAM,2BAA2B;AAEvD,OAAI,QAAQ,SAAU,QAAO,QAAQ,QAAQ;AAE7C,SAAM,YAAY;IAChB;IACA;IACA,OAAO;IACR,CAAC;;EAEL;;AAGH,eAAsB,YAAY,SAI/B;CACD,MAAM,EAAE,OAAO,UAAU;AAEzB,KAAI;EACF,MAAM,QAAQ,MAAM,gBAAgB,QAAQ,OAAO,MAAM;EAEzD,MAAM,YAAY,QAAQ,MAAM,aAAa,IAAI;EACjD,MAAM,SAAS;AAEf,MAAI,MAAM,IAAI,KACZ,OAAM,kBAAkB,OAAO,MAAM,IAAI,MAAM;GAC7C,GAAG,QAAQ;GACX;GACA,aAAa;GACb;GACA;GACD,CAAC;AAGJ,QAAM,kBAAkB,OAAO,OAAO;GACpC,GAAG,QAAQ;GACX;GACA,aAAa,QAAQ,MAAM;GAC3B;GACA;GACD,CAAC;WACM;AACR,QAAM,MAAM,UAAU"}

package/dist/commands/create.cjs

@@ -1,8 +1,8 @@
 const require_rolldown_runtime = require('../_virtual/rolldown_runtime.cjs');
 let node_fs = require("node:fs");
+let node_fs_promises = require("node:fs/promises");
 let node_path = require("node:path");
 let node_url = require("node:url");
-let node_fs_promises = require("node:fs/promises");
 let inquirer = require("inquirer");
 inquirer = require_rolldown_runtime.__toESM(inquirer);
 

package/dist/commands/create.mjs

@@ -1,7 +1,7 @@
 import { existsSync, mkdirSync, readdirSync } from "node:fs";
+import { cp } from "node:fs/promises";
 import { dirname, isAbsolute, join, relative, resolve } from "node:path";
 import { fileURLToPath } from "node:url";
-import { cp } from "node:fs/promises";
 import inquirer from "inquirer";
 
 //#region src/commands/create.ts

package/dist/commands/eval.cjs

@@ -1,10 +1,11 @@
 const require_rolldown_runtime = require('../_virtual/rolldown_runtime.cjs');
-const require_utils_load_aigne = require('../utils/load-aigne.cjs');
 const require_reporter = require('../utils/evaluation/reporter.cjs');
 const require_core = require('../utils/evaluation/core.cjs');
 const require_dataset = require('../utils/evaluation/dataset.cjs');
 const require_evaluator = require('../utils/evaluation/evaluator.cjs');
 const require_runner = require('../utils/evaluation/runner.cjs');
+const require_utils_load_aigne = require('../utils/load-aigne.cjs');
+const require_utils_view = require('../utils/view.cjs');
 let node_path = require("node:path");
 let _aigne_agent_library_utils_fs = require("@aigne/agent-library/utils/fs");
 let _aigne_core_utils_agent_utils = require("@aigne/core/utils/agent-utils");
@@ -17,7 +18,14 @@ const schema = zod.z.object({
 	dataset: zod.z.string(),
 	evaluator: zod.z.string().optional(),
 	concurrency: zod.z.number().optional(),
-	output: zod.z.string().optional()
+	output: zod.z.string().optional(),
+	json: zod.z.boolean().optional(),
+	view: zod.z.enum([
+		"default",
+		"json",
+		"llm",
+		"human"
+	]).optional()
 });
 const getResolvePath = (path) => {
 	return (0, node_path.isAbsolute)(path) ? path : (0, node_path.resolve)(process.cwd(), path);
@@ -48,6 +56,19 @@ function createEvalCommand({ aigneFilePath } = {}) {
 				type: "number",
 				describe: "Concurrency level",
 				default: 1
+			}).option("json", {
+				type: "boolean",
+				describe: "Output in JSON format",
+				default: false
+			}).option("view", {
+				type: "string",
+				describe: "Output view type (human, json, llm, default)",
+				choices: [
+					"default",
+					"json",
+					"llm",
+					"human"
+				]
 			}).help(false).version(false).strict(false);
 		},
 		handler: async (options) => {
@@ -65,10 +86,28 @@ function createEvalCommand({ aigneFilePath } = {}) {
 			let evaluatorAgent;
 			if (evaluatorName) evaluatorAgent = (0, _aigne_core_utils_agent_utils.findCliAgent)(aigne.cli, "*", evaluatorName) || aigne.agents[evaluatorName] || aigne.skills[evaluatorName] || aigne.mcpServer.agents[evaluatorName];
 			if (evaluatorAgent) evaluatorAgent.model = evaluatorAgent.model ?? aigne.model;
+			const view = require_utils_view.detectView({
+				view: parsedOptions.data.view,
+				json: parsedOptions.data.json
+			}, {}, require_utils_view.createViewContext());
 			const dataset = new require_dataset.FileDataset(resolvedDatasetPath);
 			const runner = new require_runner.DefaultRunnerWithConcurrency(agent, aigne);
 			const evaluator = new require_evaluator.LLMEvaluator(aigne, evaluatorAgent);
-			const reporters = [new require_reporter.ConsoleReporter()];
+			const reporters = [];
+			switch (view) {
+				case "json":
+					reporters.push(new require_reporter.JsonReporter());
+					break;
+				case "llm":
+					reporters.push(new require_reporter.LLMReporter());
+					break;
+				case "default":
+					reporters.push(new require_reporter.LLMReporter());
+					break;
+				default:
+					reporters.push(new require_reporter.ConsoleReporter());
+					break;
+			}
 			if (options.output) {
 				const reporter = new require_reporter.CsvReporter(getResolvePath(options.output));
 				reporters.push(reporter);

package/dist/commands/eval.mjs

@@ -1,9 +1,10 @@
-import { loadAIGNE } from "../utils/load-aigne.mjs";
-import { ConsoleReporter, CsvReporter } from "../utils/evaluation/reporter.mjs";
+import { ConsoleReporter, CsvReporter, JsonReporter, LLMReporter } from "../utils/evaluation/reporter.mjs";
 import { runEvaluationPipeline } from "../utils/evaluation/core.mjs";
 import { FileDataset } from "../utils/evaluation/dataset.mjs";
 import { LLMEvaluator } from "../utils/evaluation/evaluator.mjs";
 import { DefaultRunnerWithConcurrency } from "../utils/evaluation/runner.mjs";
+import { loadAIGNE } from "../utils/load-aigne.mjs";
+import { createViewContext, detectView } from "../utils/view.mjs";
 import { isAbsolute, resolve } from "node:path";
 import { exists } from "@aigne/agent-library/utils/fs";
 import { findCliAgent } from "@aigne/core/utils/agent-utils";
@@ -16,7 +17,14 @@ const schema = z$1.object({
 	dataset: z$1.string(),
 	evaluator: z$1.string().optional(),
 	concurrency: z$1.number().optional(),
-	output: z$1.string().optional()
+	output: z$1.string().optional(),
+	json: z$1.boolean().optional(),
+	view: z$1.enum([
+		"default",
+		"json",
+		"llm",
+		"human"
+	]).optional()
 });
 const getResolvePath = (path$1) => {
 	return isAbsolute(path$1) ? path$1 : resolve(process.cwd(), path$1);
@@ -47,6 +55,19 @@ function createEvalCommand({ aigneFilePath } = {}) {
 				type: "number",
 				describe: "Concurrency level",
 				default: 1
+			}).option("json", {
+				type: "boolean",
+				describe: "Output in JSON format",
+				default: false
+			}).option("view", {
+				type: "string",
+				describe: "Output view type (human, json, llm, default)",
+				choices: [
+					"default",
+					"json",
+					"llm",
+					"human"
+				]
 			}).help(false).version(false).strict(false);
 		},
 		handler: async (options) => {
@@ -64,10 +85,28 @@ function createEvalCommand({ aigneFilePath } = {}) {
 			let evaluatorAgent;
 			if (evaluatorName) evaluatorAgent = findCliAgent(aigne.cli, "*", evaluatorName) || aigne.agents[evaluatorName] || aigne.skills[evaluatorName] || aigne.mcpServer.agents[evaluatorName];
 			if (evaluatorAgent) evaluatorAgent.model = evaluatorAgent.model ?? aigne.model;
+			const view = detectView({
+				view: parsedOptions.data.view,
+				json: parsedOptions.data.json
+			}, {}, createViewContext());
 			const dataset = new FileDataset(resolvedDatasetPath);
 			const runner = new DefaultRunnerWithConcurrency(agent, aigne);
 			const evaluator = new LLMEvaluator(aigne, evaluatorAgent);
-			const reporters = [new ConsoleReporter()];
+			const reporters = [];
+			switch (view) {
+				case "json":
+					reporters.push(new JsonReporter());
+					break;
+				case "llm":
+					reporters.push(new LLMReporter());
+					break;
+				case "default":
+					reporters.push(new LLMReporter());
+					break;
+				default:
+					reporters.push(new ConsoleReporter());
+					break;
+			}
 			if (options.output) {
 				const reporter = new CsvReporter(getResolvePath(options.output));
 				reporters.push(reporter);

package/dist/commands/eval.mjs.map

@@ -1 +1 @@
-{"version":3,"file":"eval.mjs","names":["z","path"],"sources":["../../src/commands/eval.ts"],"sourcesContent":["import { isAbsolute, resolve } from \"node:path\";\nimport { exists } from \"@aigne/agent-library/utils/fs\";\nimport type { Agent } from \"@aigne/core\";\nimport { findCliAgent } from \"@aigne/core/utils/agent-utils\";\nimport type { CommandModule } from \"yargs\";\nimport { z } from \"zod\";\nimport { runEvaluationPipeline } from \"../utils/evaluation/core.js\";\nimport { FileDataset } from \"../utils/evaluation/dataset.js\";\nimport { LLMEvaluator } from \"../utils/evaluation/evaluator.js\";\nimport { ConsoleReporter, CsvReporter } from \"../utils/evaluation/reporter.js\";\nimport { DefaultRunnerWithConcurrency } from \"../utils/evaluation/runner.js\";\nimport { loadAIGNE } from \"../utils/load-aigne.js\";\n\nconst schema = z.object({\n  path: z.string().optional(),\n  agent: z.string(),\n  dataset: z.string(),\n  evaluator: z.string().optional(),\n  concurrency: z.number().optional(),\n  output: z.string().optional(),\n});\n\nconst getResolvePath = (path: string) => {\n  return isAbsolute(path) ? path : resolve(process.cwd(), path);\n};\n\nexport function createEvalCommand({\n  aigneFilePath,\n}: {\n  aigneFilePath?: string;\n} = {}): CommandModule<\n  unknown,\n  {\n    path?: string;\n    agent?: string;\n    dataset?: string;\n    evaluator?: string;\n    concurrency?: number;\n    output?: string;\n  }\n> {\n  return {\n    command: \"eval [path] [agent]\",\n    describe: \"Evaluate AIGNE for the specified path\",\n    builder: async (yargs) => {\n      return yargs\n        .positional(\"path\", {\n          type: \"string\",\n          describe: \"Path to the agents directory or URL to an aigne project\",\n          default: \".\",\n        })\n        .positional(\"agent\", {\n          type: \"string\",\n          describe: \"Name of the agent to evaluate\",\n        })\n        .positional(\"dataset\", {\n          type: \"string\",\n          describe: \"Path to the dataset file\",\n        })\n        .positional(\"evaluator\", {\n          type: \"string\",\n          describe: \"Name of the evaluator to use\",\n        })\n        .positional(\"output\", {\n          alias: \"o\",\n          type: \"string\",\n          describe: \"Path to the output file\",\n        })\n        .positional(\"concurrency\", {\n          type: \"number\",\n          describe: \"Concurrency level\",\n          default: 1,\n        })\n        .help(false)\n        .version(false)\n        .strict(false);\n    },\n    handler: async (options) => {\n      const parsedOptions = await schema.safeParseAsync(options);\n      if (!parsedOptions.success) {\n        throw new Error(`Invalid options: ${JSON.stringify(parsedOptions.error.format())}`);\n      }\n\n      const {\n        agent: entryAgent,\n        dataset: datasetPath,\n        evaluator: evaluatorName,\n        concurrency,\n      } = parsedOptions.data;\n\n      const path = parsedOptions.data?.path;\n      const aigne = await loadAIGNE({ path: aigneFilePath || path || \".\" });\n\n      const resolvedDatasetPath = getResolvePath(datasetPath);\n      if (!(await exists(resolvedDatasetPath))) {\n        throw new Error(\"Dataset file does not exist\");\n      }\n\n      const { chat } = aigne.cli;\n      const agent =\n        chat && chat.name === entryAgent\n          ? chat\n          : findCliAgent(aigne.cli, \"*\", entryAgent) ||\n            aigne.agents[entryAgent] ||\n            aigne.skills[entryAgent] ||\n            aigne.mcpServer.agents[entryAgent];\n\n      if (!agent) throw new Error(\"Entry agent does not exist\");\n      agent.model = agent.model ?? aigne.model;\n\n      let evaluatorAgent: Agent | undefined;\n      if (evaluatorName) {\n        evaluatorAgent =\n          findCliAgent(aigne.cli, \"*\", evaluatorName) ||\n          aigne.agents[evaluatorName] ||\n          aigne.skills[evaluatorName] ||\n          aigne.mcpServer.agents[evaluatorName];\n      }\n\n      if (evaluatorAgent) {\n        evaluatorAgent.model = evaluatorAgent.model ?? aigne.model;\n      }\n\n      const dataset = new FileDataset(resolvedDatasetPath);\n      const runner = new DefaultRunnerWithConcurrency(agent, aigne);\n      const evaluator = new LLMEvaluator(aigne, evaluatorAgent);\n      const reporters = [new ConsoleReporter()];\n      if (options.output) {\n        const resolvedReporterPath = getResolvePath(options.output);\n        const reporter = new CsvReporter(resolvedReporterPath);\n        reporters.push(reporter);\n      }\n\n      await runEvaluationPipeline({\n        dataset,\n        runner,\n        evaluators: [evaluator],\n        reporters: reporters,\n        options: { concurrency },\n      });\n    },\n  };\n}\n"],"mappings":";;;;;;;;;;;;AAaA,MAAM,SAASA,IAAE,OAAO;CACtB,MAAMA,IAAE,QAAQ,CAAC,UAAU;CAC3B,OAAOA,IAAE,QAAQ;CACjB,SAASA,IAAE,QAAQ;CACnB,WAAWA,IAAE,QAAQ,CAAC,UAAU;CAChC,aAAaA,IAAE,QAAQ,CAAC,UAAU;CAClC,QAAQA,IAAE,QAAQ,CAAC,UAAU;CAC9B,CAAC;AAEF,MAAM,kBAAkB,WAAiB;AACvC,QAAO,WAAWC,OAAK,GAAGA,SAAO,QAAQ,QAAQ,KAAK,EAAEA,OAAK;;AAG/D,SAAgB,kBAAkB,EAChC,kBAGE,EAAE,EAUJ;AACA,QAAO;EACL,SAAS;EACT,UAAU;EACV,SAAS,OAAO,UAAU;AACxB,UAAO,MACJ,WAAW,QAAQ;IAClB,MAAM;IACN,UAAU;IACV,SAAS;IACV,CAAC,CACD,WAAW,SAAS;IACnB,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,WAAW;IACrB,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,aAAa;IACvB,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,UAAU;IACpB,OAAO;IACP,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,eAAe;IACzB,MAAM;IACN,UAAU;IACV,SAAS;IACV,CAAC,CACD,KAAK,MAAM,CACX,QAAQ,MAAM,CACd,OAAO,MAAM;;EAElB,SAAS,OAAO,YAAY;GAC1B,MAAM,gBAAgB,MAAM,OAAO,eAAe,QAAQ;AAC1D,OAAI,CAAC,cAAc,QACjB,OAAM,IAAI,MAAM,oBAAoB,KAAK,UAAU,cAAc,MAAM,QAAQ,CAAC,GAAG;GAGrF,MAAM,EACJ,OAAO,YACP,SAAS,aACT,WAAW,eACX,gBACE,cAAc;GAElB,MAAMA,SAAO,cAAc,MAAM;GACjC,MAAM,QAAQ,MAAM,UAAU,EAAE,MAAM,iBAAiBA,UAAQ,KAAK,CAAC;GAErE,MAAM,sBAAsB,eAAe,YAAY;AACvD,OAAI,CAAE,MAAM,OAAO,oBAAoB,CACrC,OAAM,IAAI,MAAM,8BAA8B;GAGhD,MAAM,EAAE,SAAS,MAAM;GACvB,MAAM,QACJ,QAAQ,KAAK,SAAS,aAClB,OACA,aAAa,MAAM,KAAK,KAAK,WAAW,IACxC,MAAM,OAAO,eACb,MAAM,OAAO,eACb,MAAM,UAAU,OAAO;AAE7B,OAAI,CAAC,MAAO,OAAM,IAAI,MAAM,6BAA6B;AACzD,SAAM,QAAQ,MAAM,SAAS,MAAM;GAEnC,IAAI;AACJ,OAAI,cACF,kBACE,aAAa,MAAM,KAAK,KAAK,cAAc,IAC3C,MAAM,OAAO,kBACb,MAAM,OAAO,kBACb,MAAM,UAAU,OAAO;AAG3B,OAAI,eACF,gBAAe,QAAQ,eAAe,SAAS,MAAM;GAGvD,MAAM,UAAU,IAAI,YAAY,oBAAoB;GACpD,MAAM,SAAS,IAAI,6BAA6B,OAAO,MAAM;GAC7D,MAAM,YAAY,IAAI,aAAa,OAAO,eAAe;GACzD,MAAM,YAAY,CAAC,IAAI,iBAAiB,CAAC;AACzC,OAAI,QAAQ,QAAQ;IAElB,MAAM,WAAW,IAAI,YADQ,eAAe,QAAQ,OAAO,CACL;AACtD,cAAU,KAAK,SAAS;;AAG1B,SAAM,sBAAsB;IAC1B;IACA;IACA,YAAY,CAAC,UAAU;IACZ;IACX,SAAS,EAAE,aAAa;IACzB,CAAC;;EAEL"}
+{"version":3,"file":"eval.mjs","names":["z","path"],"sources":["../../src/commands/eval.ts"],"sourcesContent":["import { isAbsolute, resolve } from \"node:path\";\nimport { exists } from \"@aigne/agent-library/utils/fs\";\nimport type { Agent } from \"@aigne/core\";\nimport { findCliAgent } from \"@aigne/core/utils/agent-utils\";\nimport type { CommandModule } from \"yargs\";\nimport { z } from \"zod\";\nimport { runEvaluationPipeline } from \"../utils/evaluation/core.js\";\nimport { FileDataset } from \"../utils/evaluation/dataset.js\";\nimport { LLMEvaluator } from \"../utils/evaluation/evaluator.js\";\nimport {\n  ConsoleReporter,\n  CsvReporter,\n  JsonReporter,\n  LLMReporter,\n} from \"../utils/evaluation/reporter.js\";\nimport { DefaultRunnerWithConcurrency } from \"../utils/evaluation/runner.js\";\nimport { loadAIGNE } from \"../utils/load-aigne.js\";\nimport { createViewContext, detectView, type ViewType } from \"../utils/view.js\";\n\nconst schema = z.object({\n  path: z.string().optional(),\n  agent: z.string(),\n  dataset: z.string(),\n  evaluator: z.string().optional(),\n  concurrency: z.number().optional(),\n  output: z.string().optional(),\n  json: z.boolean().optional(),\n  view: z.enum([\"default\", \"json\", \"llm\", \"human\"]).optional(),\n});\n\nconst getResolvePath = (path: string) => {\n  return isAbsolute(path) ? path : resolve(process.cwd(), path);\n};\n\nexport function createEvalCommand({\n  aigneFilePath,\n}: {\n  aigneFilePath?: string;\n} = {}): CommandModule<\n  unknown,\n  {\n    path?: string;\n    agent?: string;\n    dataset?: string;\n    evaluator?: string;\n    concurrency?: number;\n    output?: string;\n  }\n> {\n  return {\n    command: \"eval [path] [agent]\",\n    describe: \"Evaluate AIGNE for the specified path\",\n    builder: async (yargs) => {\n      return yargs\n        .positional(\"path\", {\n          type: \"string\",\n          describe: \"Path to the agents directory or URL to an aigne project\",\n          default: \".\",\n        })\n        .positional(\"agent\", {\n          type: \"string\",\n          describe: \"Name of the agent to evaluate\",\n        })\n        .positional(\"dataset\", {\n          type: \"string\",\n          describe: \"Path to the dataset file\",\n        })\n        .positional(\"evaluator\", {\n          type: \"string\",\n          describe: \"Name of the evaluator to use\",\n        })\n        .positional(\"output\", {\n          alias: \"o\",\n          type: \"string\",\n          describe: \"Path to the output file\",\n        })\n        .positional(\"concurrency\", {\n          type: \"number\",\n          describe: \"Concurrency level\",\n          default: 1,\n        })\n        .option(\"json\", {\n          type: \"boolean\",\n          describe: \"Output in JSON format\",\n          default: false,\n        })\n        .option(\"view\", {\n          type: \"string\",\n          describe: \"Output view type (human, json, llm, default)\",\n          choices: [\"default\", \"json\", \"llm\", \"human\"] as const,\n        })\n        .help(false)\n        .version(false)\n        .strict(false);\n    },\n    handler: async (options) => {\n      const parsedOptions = await schema.safeParseAsync(options);\n      if (!parsedOptions.success) {\n        throw new Error(`Invalid options: ${JSON.stringify(parsedOptions.error.format())}`);\n      }\n\n      const {\n        agent: entryAgent,\n        dataset: datasetPath,\n        evaluator: evaluatorName,\n        concurrency,\n      } = parsedOptions.data;\n\n      const path = parsedOptions.data?.path;\n      const aigne = await loadAIGNE({ path: aigneFilePath || path || \".\" });\n\n      const resolvedDatasetPath = getResolvePath(datasetPath);\n      if (!(await exists(resolvedDatasetPath))) {\n        throw new Error(\"Dataset file does not exist\");\n      }\n\n      const { chat } = aigne.cli;\n      const agent =\n        chat && chat.name === entryAgent\n          ? chat\n          : findCliAgent(aigne.cli, \"*\", entryAgent) ||\n            aigne.agents[entryAgent] ||\n            aigne.skills[entryAgent] ||\n            aigne.mcpServer.agents[entryAgent];\n\n      if (!agent) throw new Error(\"Entry agent does not exist\");\n      agent.model = agent.model ?? aigne.model;\n\n      let evaluatorAgent: Agent | undefined;\n      if (evaluatorName) {\n        evaluatorAgent =\n          findCliAgent(aigne.cli, \"*\", evaluatorName) ||\n          aigne.agents[evaluatorName] ||\n          aigne.skills[evaluatorName] ||\n          aigne.mcpServer.agents[evaluatorName];\n      }\n\n      if (evaluatorAgent) {\n        evaluatorAgent.model = evaluatorAgent.model ?? aigne.model;\n      }\n\n      // Detect view based on options and context\n      const view = detectView(\n        { view: parsedOptions.data.view as ViewType, json: parsedOptions.data.json },\n        {},\n        createViewContext(),\n      );\n\n      const dataset = new FileDataset(resolvedDatasetPath);\n      const runner = new DefaultRunnerWithConcurrency(agent, aigne);\n      const evaluator = new LLMEvaluator(aigne, evaluatorAgent);\n\n      // Select reporter based on view mode\n      const reporters = [];\n      switch (view) {\n        case \"json\":\n          reporters.push(new JsonReporter());\n          break;\n        case \"llm\":\n          reporters.push(new LLMReporter());\n          break;\n        case \"default\":\n          // Machine truth: use LLM reporter for pipe-safe output\n          reporters.push(new LLMReporter());\n          break;\n        default:\n          reporters.push(new ConsoleReporter());\n          break;\n      }\n\n      if (options.output) {\n        const resolvedReporterPath = getResolvePath(options.output);\n        const reporter = new CsvReporter(resolvedReporterPath);\n        reporters.push(reporter);\n      }\n\n      await runEvaluationPipeline({\n        dataset,\n        runner,\n        evaluators: [evaluator],\n        reporters: reporters,\n        options: { concurrency },\n      });\n    },\n  };\n}\n"],"mappings":";;;;;;;;;;;;;AAmBA,MAAM,SAASA,IAAE,OAAO;CACtB,MAAMA,IAAE,QAAQ,CAAC,UAAU;CAC3B,OAAOA,IAAE,QAAQ;CACjB,SAASA,IAAE,QAAQ;CACnB,WAAWA,IAAE,QAAQ,CAAC,UAAU;CAChC,aAAaA,IAAE,QAAQ,CAAC,UAAU;CAClC,QAAQA,IAAE,QAAQ,CAAC,UAAU;CAC7B,MAAMA,IAAE,SAAS,CAAC,UAAU;CAC5B,MAAMA,IAAE,KAAK;EAAC;EAAW;EAAQ;EAAO;EAAQ,CAAC,CAAC,UAAU;CAC7D,CAAC;AAEF,MAAM,kBAAkB,WAAiB;AACvC,QAAO,WAAWC,OAAK,GAAGA,SAAO,QAAQ,QAAQ,KAAK,EAAEA,OAAK;;AAG/D,SAAgB,kBAAkB,EAChC,kBAGE,EAAE,EAUJ;AACA,QAAO;EACL,SAAS;EACT,UAAU;EACV,SAAS,OAAO,UAAU;AACxB,UAAO,MACJ,WAAW,QAAQ;IAClB,MAAM;IACN,UAAU;IACV,SAAS;IACV,CAAC,CACD,WAAW,SAAS;IACnB,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,WAAW;IACrB,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,aAAa;IACvB,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,UAAU;IACpB,OAAO;IACP,MAAM;IACN,UAAU;IACX,CAAC,CACD,WAAW,eAAe;IACzB,MAAM;IACN,UAAU;IACV,SAAS;IACV,CAAC,CACD,OAAO,QAAQ;IACd,MAAM;IACN,UAAU;IACV,SAAS;IACV,CAAC,CACD,OAAO,QAAQ;IACd,MAAM;IACN,UAAU;IACV,SAAS;KAAC;KAAW;KAAQ;KAAO;KAAQ;IAC7C,CAAC,CACD,KAAK,MAAM,CACX,QAAQ,MAAM,CACd,OAAO,MAAM;;EAElB,SAAS,OAAO,YAAY;GAC1B,MAAM,gBAAgB,MAAM,OAAO,eAAe,QAAQ;AAC1D,OAAI,CAAC,cAAc,QACjB,OAAM,IAAI,MAAM,oBAAoB,KAAK,UAAU,cAAc,MAAM,QAAQ,CAAC,GAAG;GAGrF,MAAM,EACJ,OAAO,YACP,SAAS,aACT,WAAW,eACX,gBACE,cAAc;GAElB,MAAMA,SAAO,cAAc,MAAM;GACjC,MAAM,QAAQ,MAAM,UAAU,EAAE,MAAM,iBAAiBA,UAAQ,KAAK,CAAC;GAErE,MAAM,sBAAsB,eAAe,YAAY;AACvD,OAAI,CAAE,MAAM,OAAO,oBAAoB,CACrC,OAAM,IAAI,MAAM,8BAA8B;GAGhD,MAAM,EAAE,SAAS,MAAM;GACvB,MAAM,QACJ,QAAQ,KAAK,SAAS,aAClB,OACA,aAAa,MAAM,KAAK,KAAK,WAAW,IACxC,MAAM,OAAO,eACb,MAAM,OAAO,eACb,MAAM,UAAU,OAAO;AAE7B,OAAI,CAAC,MAAO,OAAM,IAAI,MAAM,6BAA6B;AACzD,SAAM,QAAQ,MAAM,SAAS,MAAM;GAEnC,IAAI;AACJ,OAAI,cACF,kBACE,aAAa,MAAM,KAAK,KAAK,cAAc,IAC3C,MAAM,OAAO,kBACb,MAAM,OAAO,kBACb,MAAM,UAAU,OAAO;AAG3B,OAAI,eACF,gBAAe,QAAQ,eAAe,SAAS,MAAM;GAIvD,MAAM,OAAO,WACX;IAAE,MAAM,cAAc,KAAK;IAAkB,MAAM,cAAc,KAAK;IAAM,EAC5E,EAAE,EACF,mBAAmB,CACpB;GAED,MAAM,UAAU,IAAI,YAAY,oBAAoB;GACpD,MAAM,SAAS,IAAI,6BAA6B,OAAO,MAAM;GAC7D,MAAM,YAAY,IAAI,aAAa,OAAO,eAAe;GAGzD,MAAM,YAAY,EAAE;AACpB,WAAQ,MAAR;IACE,KAAK;AACH,eAAU,KAAK,IAAI,cAAc,CAAC;AAClC;IACF,KAAK;AACH,eAAU,KAAK,IAAI,aAAa,CAAC;AACjC;IACF,KAAK;AAEH,eAAU,KAAK,IAAI,aAAa,CAAC;AACjC;IACF;AACE,eAAU,KAAK,IAAI,iBAAiB,CAAC;AACrC;;AAGJ,OAAI,QAAQ,QAAQ;IAElB,MAAM,WAAW,IAAI,YADQ,eAAe,QAAQ,OAAO,CACL;AACtD,cAAU,KAAK,SAAS;;AAG1B,SAAM,sBAAsB;IAC1B;IACA;IACA,YAAY,CAAC,UAAU;IACZ;IACX,SAAS,EAAE,aAAa;IACzB,CAAC;;EAEL"}