expert_eval_cli 1.5.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -14,27 +14,32 @@ List experts (paginated).
14
14
  Query params:
15
15
  - `page` (int, default 1)
16
16
  - `pageSize` (int, default 50)
17
- - `keyword` (string, optional)
17
+ - `keyword` (string, optional; matches name/business ID/description)
18
+ - `status` (`developing` | `adjusting` | `published` | `paused`, optional)
18
19
 
19
20
  Response: `{ items: Expert[], total, page, pageSize }`
20
21
 
21
22
  ### `POST /openapi/experts`
22
- Create expert (idempotent by `expertId`).
23
+ Create an expert.
23
24
 
24
- Body: `{ expertId, name, avatar?, tags?, description?, recommendedQuestion?, aiTags?, owner?, latestVersionId? }`
25
+ Body: `{ expertId, name, owner?, description?, status? }`
25
26
 
26
- ### `PUT /openapi/experts`
27
- Update expert (idempotent by `expertId`, creates if absent).
27
+ ### `GET /openapi/experts/{expertId}`
28
+ Get expert by business ID.
28
29
 
29
- Body: same as POST, all fields required except those marked optional.
30
+ ### `PATCH /openapi/experts/{expertId}`
31
+ Partially update an expert.
30
32
 
31
- ### `GET /openapi/experts/{id}`
32
- Get expert by primary ID.
33
+ Body: `{ name?, owner?, description?, status? }`
33
34
 
34
- ### `DELETE /openapi/experts/{id}`
35
- Delete expert by primary ID.
35
+ ### `DELETE /openapi/experts/{expertId}`
36
+ Delete expert by business ID. Response: `{ id }`
36
37
 
37
- Response: `{ id }`
38
+ ### `GET /openapi/experts/{expertId}/agent-card`
39
+ Get raw `agent.json` object for the expert.
40
+
41
+ ### `GET /openapi/experts/{expertId}/agent.yml`
42
+ Get raw YAML text for the expert. Response content type: `text/yaml`.
38
43
 
39
44
  ## Versions
40
45
 
@@ -44,80 +49,81 @@ List versions (paginated).
44
49
  Query params:
45
50
  - `page` (int, default 1)
46
51
  - `pageSize` (int, default 50)
47
- - `expertId` (string, optional filter)
52
+ - `expertId` (string, optional)
48
53
 
49
54
  ### `POST /openapi/versions`
50
- Create version (idempotent by `expertId` + `hash`).
51
-
52
- Body: `{ expertId, hash, commitMessage?, author?, versionDate?, isPassed, bac?, dcg? }`
55
+ Upload Base64 encoded ZIP content and create a version. The server unzips and parses `agent.json`/`agent.yaml` plus `skills/*/SKILL.md` frontmatter, and creates evaluation tasks/records automatically. If already parsed, callers may provide parsed fields and skip server-side parsing.
53
56
 
54
- ### `PUT /openapi/versions`
55
- Update version (idempotent by `expertId` + `hash`).
56
-
57
- Body: same shape as POST.
57
+ Body: `{ expertId, packageZip?, packageName?, agentJson?, agentYaml?, skillJson? }`
58
58
 
59
59
  ### `GET /openapi/versions/{id}`
60
60
  Get version by primary ID.
61
61
 
62
62
  ### `DELETE /openapi/versions/{id}`
63
- Delete version by primary ID.
64
-
65
- Response: `{ id }`
63
+ Delete version by primary ID. Response: `{ id }`
66
64
 
67
- ## Eval Items
65
+ ## Eval Tasks
68
66
 
69
- ### `GET /openapi/eval-items`
70
- List eval items (paginated).
67
+ ### `GET /openapi/eval-tasks`
68
+ List evaluation tasks (paginated).
71
69
 
72
70
  Query params:
73
71
  - `page` (int, default 1)
74
72
  - `pageSize` (int, default 50)
73
+ - `expertId` (string, optional)
74
+ - `versionId` (string, optional)
75
+ - `status` (`pending` | `in_progress` | `failed` | `changes_requested` | `passed`, optional)
75
76
 
76
- ### `POST /openapi/eval-items`
77
- Create eval item.
77
+ ### `POST /openapi/eval-tasks`
78
+ Create an evaluation task for a version, generating all evaluation records and cases automatically.
78
79
 
79
- Body: `{ title, itemTypes?, query?, acceptanceCriteria?, isVeto? }`
80
+ Body: `{ versionId, name? }`
80
81
 
81
- `itemTypes` enum values: `structure`, `bac`, `dcg`, `process`, `deliver`.
82
+ ### `GET /openapi/eval-tasks/{id}`
83
+ Get task detail, including version raw content, all evaluation records and cases.
82
84
 
83
- ### `GET /openapi/eval-items/{id}`
84
- Get eval item by primary ID.
85
+ ### `DELETE /openapi/eval-tasks/{id}`
86
+ Delete an evaluation task. Response: `{ id }`
85
87
 
86
- ### `PUT /openapi/eval-items/{id}`
87
- Update eval item by primary ID.
88
+ ### `GET /openapi/eval-tasks/{id}/records`
89
+ Get all records for an evaluation task, including the original standard content for each item.
88
90
 
89
- Body: `{ title?, itemTypes?, query?, acceptanceCriteria?, isVeto? }`
90
-
91
- ### `DELETE /openapi/eval-items/{id}`
92
- Delete eval item by primary ID.
93
-
94
- Response: `{ id }`
91
+ ### `GET /openapi/eval-tasks/{id}/agent-card`
92
+ Get the raw agent card for the version evaluated by this task.
95
93
 
96
94
  ## Eval Records
97
95
 
98
96
  ### `GET /openapi/eval-records`
99
- List eval records (paginated).
97
+ List evaluation records by task.
100
98
 
101
99
  Query params:
102
- - `page` (int, default 1)
103
- - `pageSize` (int, default 50)
104
- - `versionId` (string, optional filter)
105
- - `evalItemId` (string, optional filter)
100
+ - `evalTaskId` (string, required)
101
+
102
+ Response: `EvalRecord[]`
103
+
104
+ ### `POST /openapi/eval-records/batch`
105
+ Batch update evaluation record status, comment and evidence.
106
106
 
107
- ### `POST /openapi/eval-records`
108
- Create eval record.
107
+ Body: `{ records: [{ id, status?, comment?, evidence? }] }`
109
108
 
110
- Body: `{ versionId, evalItemId, query?, result?, isPassed, bac?, dcg? }`
109
+ `status`: `untested` | `passed` | `warning` | `failed`
111
110
 
112
- ### `GET /openapi/eval-records/{id}`
113
- Get eval record by primary ID.
111
+ ### `PATCH /openapi/eval-records/{id}`
112
+ Update one evaluation record.
114
113
 
115
- ### `PUT /openapi/eval-records/{id}`
116
- Update eval record by primary ID.
114
+ Body: `{ status?, comment?, evidence? }`
117
115
 
118
- Body: `{ versionId?, evalItemId?, query?, result?, isPassed?, bac?, dcg? }`
116
+ ## Agent Accounts
117
+
118
+ ### `GET /openapi/agent/account`
119
+ List evaluation accounts.
120
+
121
+ Query params:
122
+ - `status` (`ready` | `running`, optional; default `ready`)
123
+ - `owner` (string, optional)
119
124
 
120
- ### `DELETE /openapi/eval-records/{id}`
121
- Delete eval record by primary ID.
125
+ ### `POST /openapi/agent/account/{agentId}/start`
126
+ Mark an account as running.
122
127
 
123
- Response: `{ id }`
128
+ ### `POST /openapi/agent/account/{agentId}/done`
129
+ Mark an account as ready/done.
@@ -1,3 +0,0 @@
1
- import { Command } from 'commander';
2
- export declare function makeEvalCasesCommand(): Command;
3
- //# sourceMappingURL=eval-cases.d.ts.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"eval-cases.d.ts","sourceRoot":"","sources":["../../src/commands/eval-cases.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAMpC,wBAAgB,oBAAoB,IAAI,OAAO,CA4G9C"}
@@ -1,136 +0,0 @@
1
- "use strict";
2
- var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
3
- if (k2 === undefined) k2 = k;
4
- var desc = Object.getOwnPropertyDescriptor(m, k);
5
- if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
6
- desc = { enumerable: true, get: function() { return m[k]; } };
7
- }
8
- Object.defineProperty(o, k2, desc);
9
- }) : (function(o, m, k, k2) {
10
- if (k2 === undefined) k2 = k;
11
- o[k2] = m[k];
12
- }));
13
- var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
14
- Object.defineProperty(o, "default", { enumerable: true, value: v });
15
- }) : function(o, v) {
16
- o["default"] = v;
17
- });
18
- var __importStar = (this && this.__importStar) || (function () {
19
- var ownKeys = function(o) {
20
- ownKeys = Object.getOwnPropertyNames || function (o) {
21
- var ar = [];
22
- for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
23
- return ar;
24
- };
25
- return ownKeys(o);
26
- };
27
- return function (mod) {
28
- if (mod && mod.__esModule) return mod;
29
- var result = {};
30
- if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
31
- __setModuleDefault(result, mod);
32
- return result;
33
- };
34
- })();
35
- Object.defineProperty(exports, "__esModule", { value: true });
36
- exports.makeEvalCasesCommand = makeEvalCasesCommand;
37
- const commander_1 = require("commander");
38
- const client = __importStar(require("../client.js"));
39
- const output_js_1 = require("../output.js");
40
- const utils_js_1 = require("../utils.js");
41
- function makeEvalCasesCommand() {
42
- const evalCases = new commander_1.Command('eval-cases').description('Manage evaluation cases');
43
- evalCases
44
- .command('list')
45
- .description('List evaluation cases')
46
- .requiredOption('--eval-task-id <id>', 'Filter by evaluation task ID')
47
- .option('--page <number>', 'Page number (starts from 1)', '1')
48
- .option('--page-size <number>', 'Page size (default: 50)', '50')
49
- .action(async (options) => {
50
- const globalOpts = evalCases.optsWithGlobals();
51
- try {
52
- const result = await client.listEvalCases({
53
- evalTaskId: options.evalTaskId,
54
- page: (0, utils_js_1.parseNumber)(options.page),
55
- pageSize: (0, utils_js_1.parseNumber)(options.pageSize),
56
- });
57
- if (globalOpts.quiet)
58
- return;
59
- const fmt = globalOpts.format === 'json' || !process.stdout.isTTY ? 'json' : 'csv';
60
- if (fmt === 'json') {
61
- (0, output_js_1.printJson)(result, globalOpts);
62
- }
63
- else {
64
- if (result.items.length === 0) {
65
- console.log('No evaluation cases found.');
66
- return;
67
- }
68
- (0, output_js_1.printCsv)(result.items, globalOpts);
69
- console.log(`\nPage ${result.page}/${Math.ceil(result.total / result.pageSize)} | Total: ${result.total}`);
70
- }
71
- }
72
- catch (err) {
73
- (0, output_js_1.printError)(err, globalOpts);
74
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
75
- return;
76
- }
77
- });
78
- evalCases
79
- .command('create')
80
- .description('Create a new evaluation case')
81
- .requiredOption('--eval-task-id <id>', 'Associated evaluation task ID')
82
- .requiredOption('--instruction <text>', 'Evaluation instruction')
83
- .requiredOption('--expected-result <text>', 'Expected result')
84
- .requiredOption('--evaluation-criteria <text>', 'Evaluation criteria')
85
- .option('--actual-result <text>', 'Actual result')
86
- .option('--artifact-link <url>', 'Artifact link')
87
- .option('--task-link <url>', 'Task link')
88
- .option('--extras <json>', 'Extras JSON object (e.g. \'{"source":"auto"}\')')
89
- .option('--execution-start-time <iso>', 'Execution start time (ISO 8601)')
90
- .option('--execution-end-time <iso>', 'Execution end time (ISO 8601)')
91
- .action(async (options) => {
92
- const globalOpts = evalCases.optsWithGlobals();
93
- try {
94
- let extras = null;
95
- if (options.extras) {
96
- extras = JSON.parse(options.extras);
97
- }
98
- const item = await client.createEvalCase({
99
- evalTaskId: options.evalTaskId,
100
- instruction: options.instruction,
101
- expectedResult: options.expectedResult,
102
- actualResult: options.actualResult || null,
103
- evaluationCriteria: options.evaluationCriteria,
104
- artifactLink: options.artifactLink || null,
105
- taskLink: options.taskLink || null,
106
- extras,
107
- executionStartTime: options.executionStartTime || null,
108
- executionEndTime: options.executionEndTime || null,
109
- });
110
- (0, output_js_1.printItem)(item, globalOpts);
111
- }
112
- catch (err) {
113
- (0, output_js_1.printError)(err, globalOpts);
114
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
115
- return;
116
- }
117
- });
118
- evalCases
119
- .command('delete')
120
- .description('Delete an evaluation case')
121
- .requiredOption('--id <id>', 'Evaluation case ID')
122
- .action(async (options) => {
123
- const globalOpts = evalCases.optsWithGlobals();
124
- try {
125
- const result = await client.deleteEvalCase(options.id);
126
- (0, output_js_1.printItem)(result, globalOpts);
127
- }
128
- catch (err) {
129
- (0, output_js_1.printError)(err, globalOpts);
130
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
131
- return;
132
- }
133
- });
134
- return evalCases;
135
- }
136
- //# sourceMappingURL=eval-cases.js.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"eval-cases.js","sourceRoot":"","sources":["../../src/commands/eval-cases.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAMA,oDA4GC;AAlHD,yCAAoC;AACpC,qDAAuC;AACvC,4CAA0E;AAC1E,0CAAsD;AAGtD,SAAgB,oBAAoB;IAClC,MAAM,SAAS,GAAG,IAAI,mBAAO,CAAC,YAAY,CAAC,CAAC,WAAW,CAAC,yBAAyB,CAAC,CAAC;IAEnF,SAAS;SACN,OAAO,CAAC,MAAM,CAAC;SACf,WAAW,CAAC,uBAAuB,CAAC;SACpC,cAAc,CAAC,qBAAqB,EAAE,8BAA8B,CAAC;SACrE,MAAM,CAAC,iBAAiB,EAAE,6BAA6B,EAAE,GAAG,CAAC;SAC7D,MAAM,CAAC,sBAAsB,EAAE,yBAAyB,EAAE,IAAI,CAAC;SAC/D,MAAM,CAAC,KAAK,EAAE,OAId,EAAE,EAAE;QACH,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,aAAa,CAAC;gBACxC,UAAU,EAAE,OAAO,CAAC,UAAU;gBAC9B,IAAI,EAAE,IAAA,sBAAW,EAAC,OAAO,CAAC,IAAI,CAAC;gBAC/B,QAAQ,EAAE,IAAA,sBAAW,EAAC,OAAO,CAAC,QAAQ,CAAC;aACxC,CAAC,CAAC;YACH,IAAI,UAAU,CAAC,KAAK;gBAAE,OAAO;YAC7B,MAAM,GAAG,GAAG,UAAU,CAAC,MAAM,KAAK,MAAM,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC;YACnF,IAAI,GAAG,KAAK,MAAM,EAAE,CAAC;gBACnB,IAAA,qBAAS,EAAC,MAAM,EAAE,UAAU,CAAC,CAAC;YAChC,CAAC;iBAAM,CAAC;gBACN,IAAI,MAAM,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBAC9B,OAAO,CAAC,GAAG,CAAC,4BAA4B,CAAC,CAAC;oBAC1C,OAAO;gBACT,CAAC;gBACD,IAAA,oBAAQ,EAAC,MAAM,CAAC,KAAK,EAAE,UAAU,CAAC,CAAC;gBACnC,OAAO,CAAC,GAAG,CAAC,UAAU,MAAM,CAAC,IAAI,IAAI,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,KAAK,GAAG,MAAM,CAAC,QAAQ,CAAC,aAAa,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC;YAC7G,CAAC;QACH,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;YAC5C,OAAO;QACT,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,SAAS;SACN,OAAO,CAAC,QAAQ,CAAC;SACjB,WAAW,CAAC,8BAA8B,CAAC;SAC3C,cAAc,CAAC,qBAAqB,EAAE,+BAA+B,CAAC;SACtE,cAAc,CAAC,sBAAsB,EAAE,wBAAwB,CAAC;SAChE,cAAc,CAAC,0BAA0B,EAAE,iBAAiB,CAAC;SAC7D,cAAc,CAAC,8BAA8B,EAAE,qBAAqB,CAAC;SACrE,MAAM,CAAC,wBAAwB,EAAE,eAAe,CAAC;SACjD,MAAM,CAAC,uBAAuB,EAAE,eAAe,CAAC;SAChD,MAAM,CAAC,mBAAmB,EAAE,WAAW,CAAC;SACxC,MAAM,CAAC,iBAAiB,EAAE,iDAAiD,CAAC;SAC5E,MAAM,CAAC,8BAA8B,EAAE,iCAAiC,CAAC;SACzE,MAAM,CAAC,4BAA4B,EAAE,+BAA+B,CAAC;SACrE,MAAM,CAAC,KAAK,EAAE,OAWd,EAAE,EAAE;QACH,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,IAAI,MAAM,GAAmC,IAAI,CAAC;YAClD,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;gBACnB,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAA4B,CAAC;YACjE,CAAC;YACD,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,cAAc,CAAC;gBACvC,UAAU,EAAE,OAAO,CAAC,UAAU;gBAC9B,WAAW,EAAE,OAAO,CAAC,WAAW;gBAChC,cAAc,EAAE,OAAO,CAAC,cAAc;gBACtC,YAAY,EAAE,OAAO,CAAC,YAAY,IAAI,IAAI;gBAC1C,kBAAkB,EAAE,OAAO,CAAC,kBAAkB;gBAC9C,YAAY,EAAE,OAAO,CAAC,YAAY,IAAI,IAAI;gBAC1C,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,IAAI;gBAClC,MAAM;gBACN,kBAAkB,EAAE,OAAO,CAAC,kBAAkB,IAAI,IAAI;gBACtD,gBAAgB,EAAE,OAAO,CAAC,gBAAgB,IAAI,IAAI;aACnD,CAAC,CAAC;YACH,IAAA,qBAAS,EAAC,IAA0C,EAAE,UAAU,CAAC,CAAC;QACpE,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;YAC5C,OAAO;QACT,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,SAAS;SACN,OAAO,CAAC,QAAQ,CAAC;SACjB,WAAW,CAAC,2BAA2B,CAAC;SACxC,cAAc,CAAC,WAAW,EAAE,oBAAoB,CAAC;SACjD,MAAM,CAAC,KAAK,EAAE,OAAuB,EAAE,EAAE;QACxC,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,cAAc,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC;YACvD,IAAA,qBAAS,EAAC,MAA4C,EAAE,UAAU,CAAC,CAAC;QACtE,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;YAC5C,OAAO;QACT,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,OAAO,SAAS,CAAC;AACnB,CAAC"}
@@ -1,3 +0,0 @@
1
- import { Command } from 'commander';
2
- export declare function makeEvalItemsCommand(): Command;
3
- //# sourceMappingURL=eval-items.d.ts.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"eval-items.d.ts","sourceRoot":"","sources":["../../src/commands/eval-items.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAcpC,wBAAgB,oBAAoB,IAAI,OAAO,CAqJ9C"}
@@ -1,204 +0,0 @@
1
- "use strict";
2
- var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
3
- if (k2 === undefined) k2 = k;
4
- var desc = Object.getOwnPropertyDescriptor(m, k);
5
- if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
6
- desc = { enumerable: true, get: function() { return m[k]; } };
7
- }
8
- Object.defineProperty(o, k2, desc);
9
- }) : (function(o, m, k, k2) {
10
- if (k2 === undefined) k2 = k;
11
- o[k2] = m[k];
12
- }));
13
- var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
14
- Object.defineProperty(o, "default", { enumerable: true, value: v });
15
- }) : function(o, v) {
16
- o["default"] = v;
17
- });
18
- var __importStar = (this && this.__importStar) || (function () {
19
- var ownKeys = function(o) {
20
- ownKeys = Object.getOwnPropertyNames || function (o) {
21
- var ar = [];
22
- for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
23
- return ar;
24
- };
25
- return ownKeys(o);
26
- };
27
- return function (mod) {
28
- if (mod && mod.__esModule) return mod;
29
- var result = {};
30
- if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
31
- __setModuleDefault(result, mod);
32
- return result;
33
- };
34
- })();
35
- Object.defineProperty(exports, "__esModule", { value: true });
36
- exports.makeEvalItemsCommand = makeEvalItemsCommand;
37
- const commander_1 = require("commander");
38
- const client = __importStar(require("../client.js"));
39
- const output_js_1 = require("../output.js");
40
- const utils_js_1 = require("../utils.js");
41
- const VALID_ITEM_TYPES = ['structure', 'bac', 'dcg', 'process', 'deliver'];
42
- function parseItemTypes(input) {
43
- if (!input)
44
- return undefined;
45
- const parts = input.split(',').map((s) => s.trim());
46
- return parts.filter((p) => VALID_ITEM_TYPES.includes(p));
47
- }
48
- function makeEvalItemsCommand() {
49
- const evalItems = new commander_1.Command('eval-items').description('Manage evaluation items');
50
- evalItems
51
- .command('list')
52
- .description('List evaluation items with pagination')
53
- .option('--page <n>', 'Page number (1-based)', '1')
54
- .option('--page-size <n>', 'Items per page', '50')
55
- .option('--category <text>', 'Filter by category')
56
- .option('--item-types <list>', `Comma-separated types: ${VALID_ITEM_TYPES.join(', ')}`)
57
- .action(async (options) => {
58
- const globalOpts = evalItems.optsWithGlobals();
59
- try {
60
- const params = {
61
- page: (0, utils_js_1.parseNumber)(options.page),
62
- pageSize: (0, utils_js_1.parseNumber)(options.pageSize),
63
- category: options.category,
64
- itemTypes: parseItemTypes(options.itemTypes),
65
- };
66
- const result = await client.listEvalItems(params);
67
- if (globalOpts.quiet)
68
- return;
69
- const fmt = globalOpts.format === 'json' || !process.stdout.isTTY ? 'json' : 'csv';
70
- if (fmt === 'json') {
71
- (0, output_js_1.printJson)(result, globalOpts);
72
- }
73
- else {
74
- if (result.items.length === 0) {
75
- console.log('No evaluation items found.');
76
- return;
77
- }
78
- (0, output_js_1.printCsv)(result.items.map((ei) => ({
79
- id: ei.id,
80
- title: ei.title,
81
- itemTypes: (ei.itemTypes || []).join(', '),
82
- isVeto: ei.isVeto ?? false,
83
- query: ei.query || '',
84
- })), globalOpts);
85
- console.log(`\nPage ${result.page}/${Math.ceil(result.total / result.pageSize)} | Total: ${result.total}`);
86
- }
87
- }
88
- catch (err) {
89
- (0, output_js_1.printError)(err, globalOpts);
90
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
91
- return;
92
- }
93
- });
94
- evalItems
95
- .command('get')
96
- .description('Get evaluation item by ID')
97
- .argument('<id>', 'Evaluation item ID')
98
- .action(async (id) => {
99
- const globalOpts = evalItems.optsWithGlobals();
100
- try {
101
- const item = await client.getEvalItem(id);
102
- (0, output_js_1.printItem)(item, globalOpts);
103
- }
104
- catch (err) {
105
- (0, output_js_1.printError)(err, globalOpts);
106
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
107
- }
108
- });
109
- evalItems
110
- .command('create')
111
- .description('Create a new evaluation item')
112
- .requiredOption('--item-code <code>', 'Unique item code (e.g. SPEC-01)')
113
- .requiredOption('--title <title>', 'Title')
114
- .option('--category <text>', 'Category (e.g. 规范评测, 功能评测, 场景评测)')
115
- .option('--description <text>', 'Description')
116
- .option('--max-score <n>', 'Maximum score', '0')
117
- .option('--item-types <list>', `Comma-separated types: ${VALID_ITEM_TYPES.join(', ')}`)
118
- .option('--query <text>', 'Evaluation query')
119
- .option('--acceptance-criteria <text>', 'Acceptance criteria')
120
- .option('--is-veto', 'Is veto item')
121
- .action(async (options) => {
122
- const globalOpts = evalItems.optsWithGlobals();
123
- try {
124
- const item = await client.createEvalItem({
125
- itemCode: options.itemCode,
126
- title: options.title,
127
- category: options.category || undefined,
128
- description: options.description || null,
129
- maxScore: options.maxScore ? (0, utils_js_1.parseNumber)(options.maxScore) : undefined,
130
- itemTypes: parseItemTypes(options.itemTypes),
131
- query: options.query || null,
132
- acceptanceCriteria: options.acceptanceCriteria || null,
133
- isVeto: !!options.isVeto,
134
- });
135
- (0, output_js_1.printItem)(item, globalOpts);
136
- }
137
- catch (err) {
138
- (0, output_js_1.printError)(err, globalOpts);
139
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
140
- return;
141
- }
142
- });
143
- evalItems
144
- .command('update')
145
- .description('Update an evaluation item')
146
- .argument('<id>', 'Evaluation item ID')
147
- .option('--item-code <code>', 'Unique item code')
148
- .option('--title <title>', 'Title')
149
- .option('--category <text>', 'Category')
150
- .option('--description <text>', 'Description')
151
- .option('--max-score <n>', 'Maximum score')
152
- .option('--item-types <list>', `Comma-separated types: ${VALID_ITEM_TYPES.join(', ')}`)
153
- .option('--query <text>', 'Evaluation query')
154
- .option('--acceptance-criteria <text>', 'Acceptance criteria')
155
- .option('--is-veto <bool>', 'Is veto item (true/false)')
156
- .action(async (id, options) => {
157
- const globalOpts = evalItems.optsWithGlobals();
158
- try {
159
- const payload = {};
160
- if (options.itemCode !== undefined)
161
- payload.itemCode = options.itemCode;
162
- if (options.title !== undefined)
163
- payload.title = options.title;
164
- if (options.category !== undefined)
165
- payload.category = options.category;
166
- if (options.description !== undefined)
167
- payload.description = options.description;
168
- if (options.maxScore !== undefined)
169
- payload.maxScore = (0, utils_js_1.parseNumber)(options.maxScore);
170
- if (options.itemTypes !== undefined)
171
- payload.itemTypes = parseItemTypes(options.itemTypes);
172
- if (options.query !== undefined)
173
- payload.query = options.query;
174
- if (options.acceptanceCriteria !== undefined)
175
- payload.acceptanceCriteria = options.acceptanceCriteria;
176
- if (options.isVeto !== undefined)
177
- payload.isVeto = options.isVeto === 'true';
178
- const item = await client.updateEvalItem(id, payload);
179
- (0, output_js_1.printItem)(item, globalOpts);
180
- }
181
- catch (err) {
182
- (0, output_js_1.printError)(err, globalOpts);
183
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
184
- return;
185
- }
186
- });
187
- evalItems
188
- .command('delete')
189
- .description('Delete an evaluation item')
190
- .argument('<id>', 'Evaluation item ID')
191
- .action(async (id) => {
192
- const globalOpts = evalItems.optsWithGlobals();
193
- try {
194
- await client.deleteEvalItem(id);
195
- (0, output_js_1.printItem)({ success: true, message: `Evaluation item ${id} deleted.` }, globalOpts);
196
- }
197
- catch (err) {
198
- (0, output_js_1.printError)(err, globalOpts);
199
- process.exitCode = utils_js_1.EXIT_CODES.GENERAL_ERROR;
200
- }
201
- });
202
- return evalItems;
203
- }
204
- //# sourceMappingURL=eval-items.js.map
@@ -1 +0,0 @@
1
- {"version":3,"file":"eval-items.js","sourceRoot":"","sources":["../../src/commands/eval-items.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAcA,oDAqJC;AAnKD,yCAAoC;AACpC,qDAAuC;AACvC,4CAA0E;AAC1E,0CAAsD;AAGtD,MAAM,gBAAgB,GAAmB,CAAC,WAAW,EAAE,KAAK,EAAE,KAAK,EAAE,SAAS,EAAE,SAAS,CAAC,CAAC;AAE3F,SAAS,cAAc,CAAC,KAAc;IACpC,IAAI,CAAC,KAAK;QAAE,OAAO,SAAS,CAAC;IAC7B,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAkB,CAAC,CAAC;IACpE,OAAO,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;AAC3D,CAAC;AAED,SAAgB,oBAAoB;IAClC,MAAM,SAAS,GAAG,IAAI,mBAAO,CAAC,YAAY,CAAC,CAAC,WAAW,CAAC,yBAAyB,CAAC,CAAC;IAEnF,SAAS;SACN,OAAO,CAAC,MAAM,CAAC;SACf,WAAW,CAAC,uCAAuC,CAAC;SACpD,MAAM,CAAC,YAAY,EAAE,uBAAuB,EAAE,GAAG,CAAC;SAClD,MAAM,CAAC,iBAAiB,EAAE,gBAAgB,EAAE,IAAI,CAAC;SACjD,MAAM,CAAC,mBAAmB,EAAE,oBAAoB,CAAC;SACjD,MAAM,CAAC,qBAAqB,EAAE,0BAA0B,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;SACtF,MAAM,CAAC,KAAK,EAAE,OAAoF,EAAE,EAAE;QACrG,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,MAAM,MAAM,GAAwB;gBAClC,IAAI,EAAE,IAAA,sBAAW,EAAC,OAAO,CAAC,IAAI,CAAC;gBAC/B,QAAQ,EAAE,IAAA,sBAAW,EAAC,OAAO,CAAC,QAAQ,CAAC;gBACvC,QAAQ,EAAE,OAAO,CAAC,QAAQ;gBAC1B,SAAS,EAAE,cAAc,CAAC,OAAO,CAAC,SAAS,CAAC;aAC7C,CAAC;YACF,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC;YAClD,IAAI,UAAU,CAAC,KAAK;gBAAE,OAAO;YAC7B,MAAM,GAAG,GAAG,UAAU,CAAC,MAAM,KAAK,MAAM,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC;YACnF,IAAI,GAAG,KAAK,MAAM,EAAE,CAAC;gBACnB,IAAA,qBAAS,EAAC,MAAM,EAAE,UAAU,CAAC,CAAC;YAChC,CAAC;iBAAM,CAAC;gBACN,IAAI,MAAM,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBAC9B,OAAO,CAAC,GAAG,CAAC,4BAA4B,CAAC,CAAC;oBAC1C,OAAO;gBACT,CAAC;gBACD,IAAA,oBAAQ,EACN,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;oBACxB,EAAE,EAAE,EAAE,CAAC,EAAE;oBACT,KAAK,EAAE,EAAE,CAAC,KAAK;oBACf,SAAS,EAAE,CAAC,EAAE,CAAC,SAAS,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC;oBAC1C,MAAM,EAAE,EAAE,CAAC,MAAM,IAAI,KAAK;oBAC1B,KAAK,EAAE,EAAE,CAAC,KAAK,IAAI,EAAE;iBACtB,CAAC,CAAC,EACH,UAAU,CACX,CAAC;gBACF,OAAO,CAAC,GAAG,CAAC,UAAU,MAAM,CAAC,IAAI,IAAI,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,KAAK,GAAG,MAAM,CAAC,QAAQ,CAAC,aAAa,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC;YAC7G,CAAC;QACH,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;YAC5C,OAAO;QACT,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,SAAS;SACN,OAAO,CAAC,KAAK,CAAC;SACd,WAAW,CAAC,2BAA2B,CAAC;SACxC,QAAQ,CAAC,MAAM,EAAE,oBAAoB,CAAC;SACtC,MAAM,CAAC,KAAK,EAAE,EAAU,EAAE,EAAE;QAC3B,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,WAAW,CAAC,EAAE,CAAC,CAAC;YAC1C,IAAA,qBAAS,EAAC,IAAI,EAAE,UAAU,CAAC,CAAC;QAC9B,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;QAC9C,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,SAAS;SACN,OAAO,CAAC,QAAQ,CAAC;SACjB,WAAW,CAAC,8BAA8B,CAAC;SAC3C,cAAc,CAAC,oBAAoB,EAAE,iCAAiC,CAAC;SACvE,cAAc,CAAC,iBAAiB,EAAE,OAAO,CAAC;SAC1C,MAAM,CAAC,mBAAmB,EAAE,kCAAkC,CAAC;SAC/D,MAAM,CAAC,sBAAsB,EAAE,aAAa,CAAC;SAC7C,MAAM,CAAC,iBAAiB,EAAE,eAAe,EAAE,GAAG,CAAC;SAC/C,MAAM,CAAC,qBAAqB,EAAE,0BAA0B,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;SACtF,MAAM,CAAC,gBAAgB,EAAE,kBAAkB,CAAC;SAC5C,MAAM,CAAC,8BAA8B,EAAE,qBAAqB,CAAC;SAC7D,MAAM,CAAC,WAAW,EAAE,cAAc,CAAC;SACnC,MAAM,CAAC,KAAK,EAAE,OAAY,EAAE,EAAE;QAC7B,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,cAAc,CAAC;gBACvC,QAAQ,EAAE,OAAO,CAAC,QAAQ;gBAC1B,KAAK,EAAE,OAAO,CAAC,KAAK;gBACpB,QAAQ,EAAE,OAAO,CAAC,QAAQ,IAAI,SAAS;gBACvC,WAAW,EAAE,OAAO,CAAC,WAAW,IAAI,IAAI;gBACxC,QAAQ,EAAE,OAAO,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAA,sBAAW,EAAC,OAAO,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,SAAS;gBACtE,SAAS,EAAE,cAAc,CAAC,OAAO,CAAC,SAAS,CAAC;gBAC5C,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,IAAI;gBAC5B,kBAAkB,EAAE,OAAO,CAAC,kBAAkB,IAAI,IAAI;gBACtD,MAAM,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM;aACzB,CAAC,CAAC;YACH,IAAA,qBAAS,EAAC,IAAI,EAAE,UAAU,CAAC,CAAC;QAC9B,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;YAC5C,OAAO;QACT,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,SAAS;SACN,OAAO,CAAC,QAAQ,CAAC;SACjB,WAAW,CAAC,2BAA2B,CAAC;SACxC,QAAQ,CAAC,MAAM,EAAE,oBAAoB,CAAC;SACtC,MAAM,CAAC,oBAAoB,EAAE,kBAAkB,CAAC;SAChD,MAAM,CAAC,iBAAiB,EAAE,OAAO,CAAC;SAClC,MAAM,CAAC,mBAAmB,EAAE,UAAU,CAAC;SACvC,MAAM,CAAC,sBAAsB,EAAE,aAAa,CAAC;SAC7C,MAAM,CAAC,iBAAiB,EAAE,eAAe,CAAC;SAC1C,MAAM,CAAC,qBAAqB,EAAE,0BAA0B,gBAAgB,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;SACtF,MAAM,CAAC,gBAAgB,EAAE,kBAAkB,CAAC;SAC5C,MAAM,CAAC,8BAA8B,EAAE,qBAAqB,CAAC;SAC7D,MAAM,CAAC,kBAAkB,EAAE,2BAA2B,CAAC;SACvD,MAAM,CAAC,KAAK,EAAE,EAAU,EAAE,OAAY,EAAE,EAAE;QACzC,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,MAAM,OAAO,GAA4B,EAAE,CAAC;YAC5C,IAAI,OAAO,CAAC,QAAQ,KAAK,SAAS;gBAAE,OAAO,CAAC,QAAQ,GAAG,OAAO,CAAC,QAAQ,CAAC;YACxE,IAAI,OAAO,CAAC,KAAK,KAAK,SAAS;gBAAE,OAAO,CAAC,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC;YAC/D,IAAI,OAAO,CAAC,QAAQ,KAAK,SAAS;gBAAE,OAAO,CAAC,QAAQ,GAAG,OAAO,CAAC,QAAQ,CAAC;YACxE,IAAI,OAAO,CAAC,WAAW,KAAK,SAAS;gBAAE,OAAO,CAAC,WAAW,GAAG,OAAO,CAAC,WAAW,CAAC;YACjF,IAAI,OAAO,CAAC,QAAQ,KAAK,SAAS;gBAAE,OAAO,CAAC,QAAQ,GAAG,IAAA,sBAAW,EAAC,OAAO,CAAC,QAAQ,CAAC,CAAC;YACrF,IAAI,OAAO,CAAC,SAAS,KAAK,SAAS;gBAAE,OAAO,CAAC,SAAS,GAAG,cAAc,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;YAC3F,IAAI,OAAO,CAAC,KAAK,KAAK,SAAS;gBAAE,OAAO,CAAC,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC;YAC/D,IAAI,OAAO,CAAC,kBAAkB,KAAK,SAAS;gBAAE,OAAO,CAAC,kBAAkB,GAAG,OAAO,CAAC,kBAAkB,CAAC;YACtG,IAAI,OAAO,CAAC,MAAM,KAAK,SAAS;gBAAE,OAAO,CAAC,MAAM,GAAG,OAAO,CAAC,MAAM,KAAK,MAAM,CAAC;YAE7E,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,cAAc,CAAC,EAAE,EAAE,OAAc,CAAC,CAAC;YAC7D,IAAA,qBAAS,EAAC,IAAI,EAAE,UAAU,CAAC,CAAC;QAC9B,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;YAC5C,OAAO;QACT,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,SAAS;SACN,OAAO,CAAC,QAAQ,CAAC;SACjB,WAAW,CAAC,2BAA2B,CAAC;SACxC,QAAQ,CAAC,MAAM,EAAE,oBAAoB,CAAC;SACtC,MAAM,CAAC,KAAK,EAAE,EAAU,EAAE,EAAE;QAC3B,MAAM,UAAU,GAAG,SAAS,CAAC,eAAe,EAAiB,CAAC;QAC9D,IAAI,CAAC;YACH,MAAM,MAAM,CAAC,cAAc,CAAC,EAAE,CAAC,CAAC;YAChC,IAAA,qBAAS,EAAC,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,mBAAmB,EAAE,WAAW,EAAE,EAAE,UAAU,CAAC,CAAC;QACtF,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAA,sBAAU,EAAC,GAAG,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,CAAC,QAAQ,GAAG,qBAAU,CAAC,aAAa,CAAC;QAC9C,CAAC;IACH,CAAC,CAAC,CAAC;IAEL,OAAO,SAAS,CAAC;AACnB,CAAC"}