@llamaventures/cli 2.4.0 → 2.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENT_BRIEFING.md +16 -0
- package/CHANGELOG.md +6 -0
- package/README.md +33 -0
- package/README.zh-CN.md +22 -0
- package/bin/llama-mcp.mjs +23 -0
- package/bin/llama.mjs +41 -0
- package/contracts/core-api.json +4 -4
- package/contracts/required-operations.json +132 -31
- package/lib/build-manifest.json +4 -4
- package/lib/client.mjs +8 -1
- package/lib/feedback.mjs +104 -0
- package/package.json +2 -2
package/AGENT_BRIEFING.md
CHANGED
|
@@ -233,3 +233,19 @@ artifact, then decide whether its content deserves Information or a Page patch.
|
|
|
233
233
|
- Wiki, admin audit, external pitch, authentication, and skill discovery are
|
|
234
234
|
separate domains. Deal Memory is also separate. None expands the four-action
|
|
235
235
|
Occam Deal space.
|
|
236
|
+
|
|
237
|
+
## Report UX friction
|
|
238
|
+
|
|
239
|
+
When a user, agent or both encounter a concrete obstacle (including a task that
|
|
240
|
+
eventually succeeds), use `llama feedback submit --title ... --body ...
|
|
241
|
+
--experienced-by user|agent|both` or MCP `feedback_submit`. Two fields suffice;
|
|
242
|
+
optional expected/steps/impact/workaround/suggestion explain the obstacle.
|
|
243
|
+
Use `llama help feedback` for JSON input and diagnostics. Versions are captured
|
|
244
|
+
when available; do not guess unknown host/model versions. Separate observation
|
|
245
|
+
from hypothesis, report once per obstacle, and never attach credentials, full
|
|
246
|
+
transcripts, file contents, raw arguments or arbitrary environment variables.
|
|
247
|
+
Only explicitly correlate a request from this task; concurrent agents share
|
|
248
|
+
local telemetry state. Retain submission_id for identical retries, read the
|
|
249
|
+
receipt with `llama feedback show` / `feedback_show`, and never let reporting
|
|
250
|
+
failure block the original task or produce recursive feedback. New tools need
|
|
251
|
+
Core API 5.9.0; do not loop if the server has not added them yet.
|
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,12 @@ this project adheres to [Semantic Versioning](https://semver.org).
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- Submit user/agent UX friction with `llama feedback submit` and MCP `feedback_submit`; read your receipt and resolution with `feedback show` / `feedback_show`.
|
|
12
|
+
- Capture CLI/build and OS/Node versions, plus available coding-agent host identity/version with explicit provenance. Unknown versions stay absent.
|
|
13
|
+
- Bounded authenticated submissions, retry IDs, credential redaction and no recursive feedback telemetry. New feedback commands require Core API 5.9.0; existing commands retain compatibility.
|
|
14
|
+
|
|
9
15
|
## [2.4.0] - 2026-09-09
|
|
10
16
|
|
|
11
17
|
### Added
|
package/README.md
CHANGED
|
@@ -228,3 +228,36 @@ These options require a Core version with source-content reads. Scanned PDFs
|
|
|
228
228
|
may require OCR; no-text, missing files, unsupported formats and denied storage
|
|
229
229
|
are explicit outcomes, not evidence that a memo has been read. MCP exposes the
|
|
230
230
|
same text reads inside `read_deal` and `wiki_read`.
|
|
231
|
+
|
|
232
|
+
## UX friction feedback
|
|
233
|
+
|
|
234
|
+
Report obstacles encountered by a user, agent or both, including tasks that
|
|
235
|
+
succeeded after confusion or unnecessary steps. Two fields are enough:
|
|
236
|
+
|
|
237
|
+
```bash
|
|
238
|
+
llama feedback submit --title "Confusing empty response" --body "Expected source text, but the successful response was empty." --experienced-by agent
|
|
239
|
+
llama feedback submit --file feedback.json
|
|
240
|
+
llama feedback show <feedback-id>
|
|
241
|
+
llama help feedback
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
`--file -` reads JSON from stdin. Optional `details` fields: expected, steps,
|
|
245
|
+
impact, workaround, suggestion. `experienced_by` defaults to both. The CLI adds
|
|
246
|
+
its version/build, OS/Node and available agent identity; MCP captures its
|
|
247
|
+
initialized host name/version. `--agent-name`, `--agent-version`, `--model` (or
|
|
248
|
+
`LLAMA_AGENT_CLIENT`, `LLAMA_AGENT_VERSION`, `LLAMA_AGENT_MODEL`) are explicit
|
|
249
|
+
reports. Missing versions stay absent; installed programs are never executed
|
|
250
|
+
to guess a running host version. Automatic metadata describes the submitting
|
|
251
|
+
process; retrospective reports can explicitly supply historical environment
|
|
252
|
+
and `occurred_at` through JSON.
|
|
253
|
+
|
|
254
|
+
Keep the returned `submission_id` for identical retries (`--submission-id`).
|
|
255
|
+
One report per obstacle per task. `--request-id` is optional and must refer to
|
|
256
|
+
that task and reporter; no global last-command association is inferred. Do not
|
|
257
|
+
include secrets, raw command arguments, environment dumps, full transcripts or
|
|
258
|
+
file contents. Known credential patterns are redacted, not a guarantee that
|
|
259
|
+
arbitrary text is secret-free. Submission failures are explicit and must not
|
|
260
|
+
block the original task or trigger recursive feedback. Calls use a 15-second
|
|
261
|
+
fetch budget and refuse redirects. Requires Core API 5.9.0, existing login and
|
|
262
|
+
write scope; receipt reads require read scope and only return your own reports.
|
|
263
|
+
MCP provides `feedback_submit` and `feedback_show` with the same behavior.
|
package/README.zh-CN.md
CHANGED
|
@@ -215,3 +215,25 @@ npm run verify:release
|
|
|
215
215
|
|
|
216
216
|
需要服务端支持正文读取。扫描 PDF 不自动 OCR;无文字、缺失文件、不支持的格式、
|
|
217
217
|
权限不足都会明确报告。MCP 在原有 `read_deal` 和 `wiki_read` 中提供相同正文读取能力。
|
|
218
|
+
|
|
219
|
+
## 用户与 Agent 的 UX friction
|
|
220
|
+
|
|
221
|
+
即使任务最终成功,困惑、多余步骤、反复尝试也值得反馈:
|
|
222
|
+
|
|
223
|
+
```bash
|
|
224
|
+
llama feedback submit --title "成功响应没有正文" --body "用户想总结附件,但读取成功后没有返回文本,任务无法继续。" --experienced-by agent
|
|
225
|
+
llama feedback submit --file feedback.json
|
|
226
|
+
llama feedback show <反馈ID>
|
|
227
|
+
llama help feedback
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
标题和描述必填;`experienced_by` 为 user / agent / both,默认 both。
|
|
231
|
+
CLI 自动附带版本、构建、OS/Node 和可确认的 Agent 身份;MCP 读取宿主声明的名称及版本。
|
|
232
|
+
Agent 版本未知时不猜测。`--agent-name`、`--agent-version`、`--model` 可明确补充信息。
|
|
233
|
+
较早发生的问题可通过 JSON 提供历史环境和 `occurred_at`,自动采集的环境属于当前提交进程。
|
|
234
|
+
可选 details 字段为 expected、steps、impact、workaround、suggestion。
|
|
235
|
+
同一次重试复用 submission_id;同一任务的同一障碍只报一次。
|
|
236
|
+
不附完整对话、文件、命令参数或凭证,不自动关联全局最后一次调用。
|
|
237
|
+
提交失败明确报错,不能声称成功,也不能触发递归反馈或阻塞原任务。
|
|
238
|
+
新命令需要 Core API 5.9.0,沿用现有登录;用户只能读取自己的反馈。
|
|
239
|
+
MCP 对应 feedback_submit / feedback_show。
|
package/bin/llama-mcp.mjs
CHANGED
|
@@ -26,6 +26,7 @@ import {
|
|
|
26
26
|
compactDealWriteResult,
|
|
27
27
|
prepareDealCommand,
|
|
28
28
|
} from "../lib/deal-actions.mjs";
|
|
29
|
+
import { submitFeedback, readFeedback } from "../lib/feedback.mjs";
|
|
29
30
|
import { buildDealMemoryPath } from "../lib/deal-memory-actions.mjs";
|
|
30
31
|
import { buildPageSchemaPath, MAX_PAGE_SCHEMA_FIELDS } from "../lib/page-schema.mjs";
|
|
31
32
|
|
|
@@ -51,6 +52,28 @@ async function callApi(method, path, body, transform = (value) => value) {
|
|
|
51
52
|
}
|
|
52
53
|
}
|
|
53
54
|
|
|
55
|
+
server.registerTool("feedback_submit", {
|
|
56
|
+
description: "Report concrete UX friction experienced by a user, agent or both, even if the task succeeded. Separate observations from guesses. One report per obstacle; never include secrets, transcripts, files or raw command arguments. Failures must not block the original task or trigger recursive reporting. Reuse submission_id for retries. Host identity/version is captured when available; unknown versions stay absent.",
|
|
57
|
+
inputSchema: {
|
|
58
|
+
title: z.string().min(1).max(200), body: z.string().min(1).max(8000),
|
|
59
|
+
experienced_by: z.enum(["user", "agent", "both"]).optional(), submission_id: z.string().uuid().optional(),
|
|
60
|
+
occurred_at: z.string().datetime({ offset: true }).optional(),
|
|
61
|
+
details: z.object({ expected: z.string().max(2000).optional(), steps: z.string().max(4000).optional(), impact: z.string().max(2000).optional(), workaround: z.string().max(2000).optional(), suggestion: z.string().max(2000).optional() }).strict().optional(),
|
|
62
|
+
environment: z.object({ agent_name: z.string().max(120).optional(), agent_version: z.string().max(120).optional(), model: z.string().max(120).optional(), command: z.string().max(160).optional(), error_code: z.string().max(120).optional(), request_id: z.string().max(128).optional() }).strict().optional(),
|
|
63
|
+
},
|
|
64
|
+
}, async input => {
|
|
65
|
+
try { return textResult(JSON.stringify(await submitFeedback(input, server.server.getClientVersion()), null, 2)); }
|
|
66
|
+
catch (error) { return textResult(formatErrorForDisplay(error), true); }
|
|
67
|
+
});
|
|
68
|
+
server.registerTool("feedback_show", {
|
|
69
|
+
description: "Read your own submitted friction report and resolution using its feedback ID.",
|
|
70
|
+
inputSchema: { id: z.string().uuid() },
|
|
71
|
+
annotations: { readOnlyHint: true },
|
|
72
|
+
}, async ({ id }) => {
|
|
73
|
+
try { return textResult(JSON.stringify(await readFeedback(id), null, 2)); }
|
|
74
|
+
catch (error) { return textResult(formatErrorForDisplay(error), true); }
|
|
75
|
+
});
|
|
76
|
+
|
|
54
77
|
const originSchema = z.object({
|
|
55
78
|
kind: z.enum(["user", "agent", "system"]),
|
|
56
79
|
originalUserUtterance: z.string().min(1).optional(),
|
package/bin/llama.mjs
CHANGED
|
@@ -44,6 +44,7 @@ import {
|
|
|
44
44
|
buildDealMemoryPath,
|
|
45
45
|
readMarkdownInput,
|
|
46
46
|
} from "../lib/deal-memory-actions.mjs";
|
|
47
|
+
import { submitFeedback, readFeedback, readFeedbackInput } from "../lib/feedback.mjs";
|
|
47
48
|
import { buildPageSchemaPath } from "../lib/page-schema.mjs";
|
|
48
49
|
|
|
49
50
|
const requireFromHere = createRequire(import.meta.url);
|
|
@@ -74,6 +75,8 @@ Separate preserved domains:
|
|
|
74
75
|
llama deal read <dealId> --artifact <artifactId> [--version N] [--offset N --sha256 HASH] [--limit N] [--output <file>]
|
|
75
76
|
llama wiki read <slug> --format text [--attachment <referenceId>] [--offset N --sha256 HASH] [--output <file>]
|
|
76
77
|
llama wiki search|read|save|delete|restore
|
|
78
|
+
llama feedback submit --title "..." --body "..." [--experienced-by user|agent|both]
|
|
79
|
+
llama feedback show <id>
|
|
77
80
|
llama admin auth-events|deal-events|agent-events
|
|
78
81
|
llama pitch start|say|upload|status|finalize|end
|
|
79
82
|
|
|
@@ -176,6 +179,23 @@ Start with list. Before page.patch, read only the exact fields being changed;
|
|
|
176
179
|
use a section read only when the write genuinely spans that section. This is a
|
|
177
180
|
schema/field-prompt surface, not a fifth Deal action and not Page data.`;
|
|
178
181
|
|
|
182
|
+
const HELP_FEEDBACK = `Report concrete user or agent UX friction, even when the task succeeds.
|
|
183
|
+
|
|
184
|
+
llama feedback submit --title "..." --body "..." [--experienced-by user|agent|both]
|
|
185
|
+
llama feedback submit --file <feedback.json|->
|
|
186
|
+
llama feedback show <id>
|
|
187
|
+
|
|
188
|
+
Optional flags: --expected, --steps, --impact, --workaround, --suggestion,
|
|
189
|
+
--agent-name, --agent-version, --model, --command, --error-code, --request-id,
|
|
190
|
+
--occurred-at (ISO timestamp), --submission-id (reuse for retries).
|
|
191
|
+
|
|
192
|
+
CLI/build, OS/Node and available agent identity are added automatically. Unknown
|
|
193
|
+
agent versions stay absent. LLAMA_AGENT_CLIENT, LLAMA_AGENT_VERSION and
|
|
194
|
+
LLAMA_AGENT_MODEL are explicit declarations, not inferred installed versions.
|
|
195
|
+
Only attach a request ID from this task. Do not include credentials, full
|
|
196
|
+
transcripts, file contents or raw command arguments. Report once per obstacle;
|
|
197
|
+
feedback failure must not block the task or trigger recursive feedback.`;
|
|
198
|
+
|
|
179
199
|
function parseFlags(args, allowed = null) {
|
|
180
200
|
const flags = {};
|
|
181
201
|
const positional = [];
|
|
@@ -206,6 +226,7 @@ function usage(area) {
|
|
|
206
226
|
area === "deal" ? HELP_DEAL :
|
|
207
227
|
area === "memory" ? HELP_MEMORY :
|
|
208
228
|
area === "page-schema" ? HELP_PAGE_SCHEMA :
|
|
229
|
+
area === "feedback" ? HELP_FEEDBACK :
|
|
209
230
|
HELP_ROOT,
|
|
210
231
|
);
|
|
211
232
|
}
|
|
@@ -529,6 +550,26 @@ async function main() {
|
|
|
529
550
|
return;
|
|
530
551
|
}
|
|
531
552
|
|
|
553
|
+
if (area === "feedback") {
|
|
554
|
+
if (action === "show" && rest.length === 1) { print(await readFeedback(rest[0])); return; }
|
|
555
|
+
if (action !== "submit") throw new Error("Usage: llama feedback submit|show; see llama help feedback");
|
|
556
|
+
const allowed = ["file", "title", "body", "experienced-by", "expected", "steps", "impact", "workaround", "suggestion", "agent-name", "agent-version", "model", "command", "error-code", "request-id", "occurred-at", "submission-id"];
|
|
557
|
+
const { flags, positional } = parseFlags(rest, allowed);
|
|
558
|
+
if (positional.length || Object.values(flags).some(v => typeof v !== "string")) throw new Error("Feedback flags require values; see llama help feedback");
|
|
559
|
+
let input;
|
|
560
|
+
if (flags.file) {
|
|
561
|
+
if (Object.keys(flags).some(k => !["file", "submission-id"].includes(k))) throw new Error("Use --file or field flags, not both");
|
|
562
|
+
input = await readFeedbackInput(flags.file);
|
|
563
|
+
if (flags["submission-id"]) input.submission_id = flags["submission-id"];
|
|
564
|
+
} else {
|
|
565
|
+
input = { title: flags.title, body: flags.body, details: {}, environment: {} };
|
|
566
|
+
for (const key of ["experienced-by", "occurred-at", "submission-id"]) if (flags[key]) input[key.replaceAll("-", "_")] = flags[key];
|
|
567
|
+
for (const key of ["expected", "steps", "impact", "workaround", "suggestion"]) if (flags[key]) input.details[key] = flags[key];
|
|
568
|
+
for (const key of ["agent-name", "agent-version", "model", "command", "error-code", "request-id"]) if (flags[key]) input.environment[key.replaceAll("-", "_")] = flags[key];
|
|
569
|
+
}
|
|
570
|
+
print(await submitFeedback(input)); return;
|
|
571
|
+
}
|
|
572
|
+
|
|
532
573
|
assertDealAction(area, action);
|
|
533
574
|
|
|
534
575
|
if (await handleAuth(area, action, rest)) return;
|
package/contracts/core-api.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"format": "llama.core-api-contract.v1",
|
|
3
3
|
"name": "llama-core-api",
|
|
4
|
-
"apiVersion": "5.
|
|
4
|
+
"apiVersion": "5.9.0",
|
|
5
5
|
"openapiVersion": "3.0.3",
|
|
6
|
-
"sha256": "
|
|
7
|
-
"pathCount":
|
|
8
|
-
"operationCount":
|
|
6
|
+
"sha256": "619d4cabf0fa078bab2cdbb669fc9066701fc68ce194809a37c00a431bb66d35",
|
|
7
|
+
"pathCount": 117,
|
|
8
|
+
"operationCount": 144
|
|
9
9
|
}
|
|
@@ -2,36 +2,137 @@
|
|
|
2
2
|
"format": "llama.cli-core-api-operations.v1",
|
|
3
3
|
"coreApiContract": "contracts/core-api.json",
|
|
4
4
|
"requiredOperations": [
|
|
5
|
-
{
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
{
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
{
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
{
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
{
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
{
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
{
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
{
|
|
34
|
-
|
|
35
|
-
|
|
5
|
+
{
|
|
6
|
+
"method": "DELETE",
|
|
7
|
+
"path": "/api/wiki/{slug}"
|
|
8
|
+
},
|
|
9
|
+
{
|
|
10
|
+
"method": "GET",
|
|
11
|
+
"path": "/api/admin/agent-events"
|
|
12
|
+
},
|
|
13
|
+
{
|
|
14
|
+
"method": "GET",
|
|
15
|
+
"path": "/api/admin/auth-events"
|
|
16
|
+
},
|
|
17
|
+
{
|
|
18
|
+
"method": "GET",
|
|
19
|
+
"path": "/api/admin/deal-events"
|
|
20
|
+
},
|
|
21
|
+
{
|
|
22
|
+
"method": "GET",
|
|
23
|
+
"path": "/api/agent/briefing"
|
|
24
|
+
},
|
|
25
|
+
{
|
|
26
|
+
"method": "GET",
|
|
27
|
+
"path": "/api/agent/explain"
|
|
28
|
+
},
|
|
29
|
+
{
|
|
30
|
+
"method": "GET",
|
|
31
|
+
"path": "/api/agent/manifest"
|
|
32
|
+
},
|
|
33
|
+
{
|
|
34
|
+
"method": "GET",
|
|
35
|
+
"path": "/api/agent/page-schema"
|
|
36
|
+
},
|
|
37
|
+
{
|
|
38
|
+
"method": "GET",
|
|
39
|
+
"path": "/api/agent/preferences"
|
|
40
|
+
},
|
|
41
|
+
{
|
|
42
|
+
"method": "GET",
|
|
43
|
+
"path": "/api/agent/skills"
|
|
44
|
+
},
|
|
45
|
+
{
|
|
46
|
+
"method": "GET",
|
|
47
|
+
"path": "/api/agent/skills/{slug}"
|
|
48
|
+
},
|
|
49
|
+
{
|
|
50
|
+
"method": "GET",
|
|
51
|
+
"path": "/api/deal-memory/{dealId}/story"
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
"method": "GET",
|
|
55
|
+
"path": "/api/feedback/{id}"
|
|
56
|
+
},
|
|
57
|
+
{
|
|
58
|
+
"method": "GET",
|
|
59
|
+
"path": "/api/me"
|
|
60
|
+
},
|
|
61
|
+
{
|
|
62
|
+
"method": "GET",
|
|
63
|
+
"path": "/api/oauth/authorize"
|
|
64
|
+
},
|
|
65
|
+
{
|
|
66
|
+
"method": "GET",
|
|
67
|
+
"path": "/api/occam/deals"
|
|
68
|
+
},
|
|
69
|
+
{
|
|
70
|
+
"method": "GET",
|
|
71
|
+
"path": "/api/occam/deals/{dealId}"
|
|
72
|
+
},
|
|
73
|
+
{
|
|
74
|
+
"method": "GET",
|
|
75
|
+
"path": "/api/occam/deals/{dealId}/artifacts/{artifactId}"
|
|
76
|
+
},
|
|
77
|
+
{
|
|
78
|
+
"method": "GET",
|
|
79
|
+
"path": "/api/wiki/search"
|
|
80
|
+
},
|
|
81
|
+
{
|
|
82
|
+
"method": "GET",
|
|
83
|
+
"path": "/api/wiki/{slug}"
|
|
84
|
+
},
|
|
85
|
+
{
|
|
86
|
+
"method": "PATCH",
|
|
87
|
+
"path": "/api/agent/preferences/{id}"
|
|
88
|
+
},
|
|
89
|
+
{
|
|
90
|
+
"method": "POST",
|
|
91
|
+
"path": "/api/agent/client-events"
|
|
92
|
+
},
|
|
93
|
+
{
|
|
94
|
+
"method": "POST",
|
|
95
|
+
"path": "/api/agent/preferences"
|
|
96
|
+
},
|
|
97
|
+
{
|
|
98
|
+
"method": "POST",
|
|
99
|
+
"path": "/api/external/chat"
|
|
100
|
+
},
|
|
101
|
+
{
|
|
102
|
+
"method": "POST",
|
|
103
|
+
"path": "/api/external/start-session"
|
|
104
|
+
},
|
|
105
|
+
{
|
|
106
|
+
"method": "POST",
|
|
107
|
+
"path": "/api/external/upload"
|
|
108
|
+
},
|
|
109
|
+
{
|
|
110
|
+
"method": "POST",
|
|
111
|
+
"path": "/api/feedback"
|
|
112
|
+
},
|
|
113
|
+
{
|
|
114
|
+
"method": "POST",
|
|
115
|
+
"path": "/api/oauth/revoke"
|
|
116
|
+
},
|
|
117
|
+
{
|
|
118
|
+
"method": "POST",
|
|
119
|
+
"path": "/api/oauth/token"
|
|
120
|
+
},
|
|
121
|
+
{
|
|
122
|
+
"method": "POST",
|
|
123
|
+
"path": "/api/occam/deals/commands"
|
|
124
|
+
},
|
|
125
|
+
{
|
|
126
|
+
"method": "POST",
|
|
127
|
+
"path": "/api/wiki/save"
|
|
128
|
+
},
|
|
129
|
+
{
|
|
130
|
+
"method": "POST",
|
|
131
|
+
"path": "/api/wiki/{slug}/restore"
|
|
132
|
+
},
|
|
133
|
+
{
|
|
134
|
+
"method": "PUT",
|
|
135
|
+
"path": "/api/deal-memory/{dealId}/story"
|
|
136
|
+
}
|
|
36
137
|
]
|
|
37
138
|
}
|
package/lib/build-manifest.json
CHANGED
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
{
|
|
2
2
|
"format": "llama.cli-build.v1",
|
|
3
3
|
"packageName": "@llamaventures/cli",
|
|
4
|
-
"packageVersion": "2.
|
|
5
|
-
"sourceSha": "
|
|
4
|
+
"packageVersion": "2.5.0",
|
|
5
|
+
"sourceSha": "2f1d1c40ee4a970fb8c028428aa25c2976222486",
|
|
6
6
|
"sourceKind": "github",
|
|
7
7
|
"sourceDirty": false,
|
|
8
8
|
"coreApiContract": {
|
|
9
9
|
"format": "llama.core-api-contract.v1",
|
|
10
10
|
"name": "llama-core-api",
|
|
11
|
-
"apiVersion": "5.
|
|
11
|
+
"apiVersion": "5.9.0",
|
|
12
12
|
"openapiVersion": "3.0.3",
|
|
13
|
-
"sha256": "
|
|
13
|
+
"sha256": "619d4cabf0fa078bab2cdbb669fc9066701fc68ce194809a37c00a431bb66d35"
|
|
14
14
|
}
|
|
15
15
|
}
|
package/lib/client.mjs
CHANGED
|
@@ -148,6 +148,11 @@ export function setClientRuntime(opts = {}) {
|
|
|
148
148
|
if (opts.agentClient) runtimeAgentClient = String(opts.agentClient);
|
|
149
149
|
}
|
|
150
150
|
|
|
151
|
+
export function getFeedbackRuntime() {
|
|
152
|
+
const agent = detectAgentClient();
|
|
153
|
+
return { client: runtimeClient, agent, agentSource: runtimeAgentClient || process.env.LLAMA_AGENT_CLIENT ? "reported" : agent === "unknown" ? "unknown" : "detected" };
|
|
154
|
+
}
|
|
155
|
+
|
|
151
156
|
function detectAgentClient() {
|
|
152
157
|
if (runtimeAgentClient) return runtimeAgentClient;
|
|
153
158
|
if (process.env.LLAMA_AGENT_CLIENT) return process.env.LLAMA_AGENT_CLIENT;
|
|
@@ -361,7 +366,7 @@ function summarizeResult(data) {
|
|
|
361
366
|
function shouldSkipTelemetry(endpoint) {
|
|
362
367
|
if (process.env.LLAMA_TELEMETRY === "0") return true;
|
|
363
368
|
const pathname = parseEndpoint(endpoint)?.pathname || endpoint;
|
|
364
|
-
return pathname === "/api/agent/client-events" || pathname === "/api/agent/eval-feedback";
|
|
369
|
+
return pathname === "/api/feedback" || pathname.startsWith("/api/feedback/") || pathname === "/api/agent/client-events" || pathname === "/api/agent/eval-feedback";
|
|
365
370
|
}
|
|
366
371
|
|
|
367
372
|
async function recordClientTelemetry({
|
|
@@ -602,6 +607,8 @@ async function requestWithRetry(method, endpoint, body, opts, allowRetry) {
|
|
|
602
607
|
const command = inferCommand(method, endpoint);
|
|
603
608
|
const start = Date.now();
|
|
604
609
|
const res = await fetch(`${getBaseUrl()}${endpoint}`, {
|
|
610
|
+
signal: opts.signal,
|
|
611
|
+
redirect: opts.redirect,
|
|
605
612
|
method,
|
|
606
613
|
headers: {
|
|
607
614
|
"Content-Type": "application/json",
|
package/lib/feedback.mjs
ADDED
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
import os from "node:os";
|
|
2
|
+
import { createReadStream } from "node:fs";
|
|
3
|
+
import { randomUUID } from "node:crypto";
|
|
4
|
+
import { getBuildInfo } from "./build-info.mjs";
|
|
5
|
+
import { getFeedbackRuntime, request } from "./client.mjs";
|
|
6
|
+
|
|
7
|
+
const TOP = ["submission_id", "title", "body", "experienced_by", "occurred_at", "details", "environment"];
|
|
8
|
+
const DETAILS = { expected: 2000, steps: 4000, impact: 2000, workaround: 2000, suggestion: 2000 };
|
|
9
|
+
const ENV = { surface: 12, cli_version: 80, cli_source_sha: 80, agent_name: 120, agent_version: 120,
|
|
10
|
+
agent_name_source: 12, agent_version_source: 12, model: 120, model_source: 12,
|
|
11
|
+
os: 80, os_version: 120, arch: 40, node_version: 80, command: 160, error_code: 120, request_id: 128 };
|
|
12
|
+
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
|
|
13
|
+
export function redactFeedbackText(text) {
|
|
14
|
+
return text.replace(/\b(?:llc_|sk-|ghp_|github_pat_|ya29\.)[A-Za-z0-9_.-]+/g, "[redacted]")
|
|
15
|
+
.replace(/\beyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+/g, "[redacted]")
|
|
16
|
+
.replace(/\b(Bearer)\s+[^\s,;\x22\x27]+/gi, "$1 [redacted]")
|
|
17
|
+
.replace(/\b((?:access[_-]?token|refresh[_-]?token|api[_-]?key|token|secret|password|authorization|cookie)[\x22\x27]?\s*[:=]\s*)[^\r\n]+/gi, "$1[redacted]")
|
|
18
|
+
.replace(/(https?:\/\/)[^\s/@]+:[^\s/@]+@/gi, "$1[redacted]@");
|
|
19
|
+
}
|
|
20
|
+
function object(value, allowed, name) {
|
|
21
|
+
if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error(`${name} must be an object`);
|
|
22
|
+
for (const key of Object.keys(value)) if (!allowed.includes(key)) throw new Error(`Unknown ${name} field: ${key}`);
|
|
23
|
+
return value;
|
|
24
|
+
}
|
|
25
|
+
function text(value, max, name) {
|
|
26
|
+
if (typeof value !== "string" || !value.trim() || value.trim().length > max) throw new Error(`${name} requires 1–${max} characters`);
|
|
27
|
+
return redactFeedbackText(value.trim());
|
|
28
|
+
}
|
|
29
|
+
function strings(value, fields, name) {
|
|
30
|
+
const result = {};
|
|
31
|
+
for (const [key, val] of Object.entries(object(value, Object.keys(fields), name))) result[key] = text(val, fields[key], `${name}.${key}`);
|
|
32
|
+
return result;
|
|
33
|
+
}
|
|
34
|
+
export function feedbackId(id) {
|
|
35
|
+
if (typeof id !== "string" || !UUID.test(id)) throw new Error("Feedback ID must be a UUID");
|
|
36
|
+
return id;
|
|
37
|
+
}
|
|
38
|
+
export async function readFeedbackInput(file, stdin = process.stdin) {
|
|
39
|
+
const stream = file === "-" ? stdin : createReadStream(file);
|
|
40
|
+
let size = 0; const chunks = [];
|
|
41
|
+
for await (const chunk of stream) {
|
|
42
|
+
const bytes = Buffer.from(chunk); size += bytes.length;
|
|
43
|
+
if (size > 32768) { stream.destroy(); throw new Error("Feedback exceeds 32 KiB"); }
|
|
44
|
+
chunks.push(bytes);
|
|
45
|
+
}
|
|
46
|
+
try { return JSON.parse(Buffer.concat(chunks).toString("utf8")); }
|
|
47
|
+
catch { throw new Error("Feedback file must contain valid JSON"); }
|
|
48
|
+
}
|
|
49
|
+
export function prepareFeedback(input, host) {
|
|
50
|
+
object(input, TOP, "feedback");
|
|
51
|
+
const runtime = getFeedbackRuntime(); const build = getBuildInfo();
|
|
52
|
+
const supplied = strings(input.environment ?? {}, ENV, "environment");
|
|
53
|
+
const environment = {
|
|
54
|
+
surface: runtime.client, cli_version: build.packageVersion, cli_source_sha: build.sourceSha,
|
|
55
|
+
os: os.platform(), os_version: os.release(), arch: os.arch(), node_version: process.version,
|
|
56
|
+
agent_name: runtime.agent, agent_name_source: runtime.agentSource,
|
|
57
|
+
...(host?.name ? { agent_name: host.name, agent_name_source: "host" } : {}),
|
|
58
|
+
...(host?.version ? { agent_version: host.version, agent_version_source: "host" } : {}),
|
|
59
|
+
...(process.env.LLAMA_AGENT_VERSION ? { agent_version: process.env.LLAMA_AGENT_VERSION, agent_version_source: "reported" } : {}),
|
|
60
|
+
...(process.env.LLAMA_AGENT_MODEL ? { model: process.env.LLAMA_AGENT_MODEL, model_source: "reported" } : {}),
|
|
61
|
+
...supplied,
|
|
62
|
+
};
|
|
63
|
+
if (supplied.agent_name && host?.name && supplied.agent_name !== host.name && !supplied.agent_version && !process.env.LLAMA_AGENT_VERSION) {
|
|
64
|
+
delete environment.agent_version;
|
|
65
|
+
delete environment.agent_version_source;
|
|
66
|
+
}
|
|
67
|
+
// Explicit overrides are declarations, not observations of the running host.
|
|
68
|
+
for (const key of ["agent_name", "agent_version", "model"]) {
|
|
69
|
+
if (supplied[key] && !supplied[`${key}_source`]) environment[`${key}_source`] = "reported";
|
|
70
|
+
}
|
|
71
|
+
for (const key of ["agent_name_source", "agent_version_source", "model_source"]) {
|
|
72
|
+
if (environment[key] && !["detected", "host", "reported", "unknown"].includes(environment[key])) throw new Error(`Invalid ${key}`);
|
|
73
|
+
}
|
|
74
|
+
if (!["cli", "mcp", "web", "api"].includes(environment.surface)) throw new Error("Invalid environment.surface");
|
|
75
|
+
const experiencedBy = input.experienced_by ?? "both";
|
|
76
|
+
if (!["user", "agent", "both"].includes(experiencedBy)) throw new Error("experienced_by must be user, agent or both");
|
|
77
|
+
if (input.occurred_at && (typeof input.occurred_at !== "string" || !/^\d{4}-\d\d-\d\dT.*(?:Z|[+-]\d\d:\d\d)$/.test(input.occurred_at) || !Number.isFinite(Date.parse(input.occurred_at)))) throw new Error("occurred_at must be an ISO timestamp with timezone");
|
|
78
|
+
const result = {
|
|
79
|
+
submission_id: input.submission_id ? feedbackId(input.submission_id) : randomUUID(),
|
|
80
|
+
title: text(input.title, 200, "title"), body: text(input.body, 8000, "body"), experienced_by: experiencedBy,
|
|
81
|
+
...(input.occurred_at ? { occurred_at: input.occurred_at } : {}),
|
|
82
|
+
details: strings(input.details ?? {}, DETAILS, "details"), environment: strings(environment, ENV, "environment"),
|
|
83
|
+
};
|
|
84
|
+
if (Buffer.byteLength(JSON.stringify(result)) > 32768) throw new Error("Feedback exceeds 32 KiB");
|
|
85
|
+
return result;
|
|
86
|
+
}
|
|
87
|
+
export async function submitFeedback(input, host) {
|
|
88
|
+
const body = prepareFeedback(input, host);
|
|
89
|
+
const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), 15000);
|
|
90
|
+
try {
|
|
91
|
+
return await request("POST", "/api/feedback", body, { signal: controller.signal, redirect: "error" });
|
|
92
|
+
} catch (error) {
|
|
93
|
+
const wrapped = new Error(`Error[FEEDBACK_NOT_CONFIRMED]: Feedback submission is not confirmed. ${redactFeedbackText(error.message)}\nRetry the same report with submission_id=${body.submission_id}. Do not recursively report this failure.`);
|
|
94
|
+
wrapped.code = "FEEDBACK_NOT_CONFIRMED";
|
|
95
|
+
throw wrapped;
|
|
96
|
+
} finally { clearTimeout(timer); }
|
|
97
|
+
}
|
|
98
|
+
export async function readFeedback(id) {
|
|
99
|
+
const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), 15000);
|
|
100
|
+
try {
|
|
101
|
+
// @core-api-operation GET /api/feedback/{id}
|
|
102
|
+
return await request("GET", `/api/feedback/${feedbackId(id)}`, undefined, { signal: controller.signal, redirect: "error" });
|
|
103
|
+
} finally { clearTimeout(timer); }
|
|
104
|
+
}
|
package/package.json
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@llamaventures/cli",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.5.0",
|
|
4
4
|
"description": "CLI + MCP server for the Llama Ventures investment workbench (command.llamaventures.vc).",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"scripts": {
|
|
7
7
|
"test": "npm run test:agent-routing && npm run test:contract",
|
|
8
8
|
"test:agent-routing": "node scripts/verify-agent-routing.mjs",
|
|
9
|
-
"test:contract": "node --test scripts/content-read.test.mjs scripts/build-manifest.test.mjs scripts/core-api-call-sites.test.mjs scripts/deal-actions-cli.test.mjs scripts/deal-actions.test.mjs scripts/deal-memory-actions-cli.test.mjs scripts/deal-memory-actions.test.mjs scripts/mcp-four-action-surface.test.mjs scripts/page-schema-cli.test.mjs scripts/page-schema.test.mjs scripts/server-compatibility.test.mjs scripts/verify-core-api-contract.test.mjs && node scripts/verify-core-api-contract.mjs",
|
|
9
|
+
"test:contract": "node --test scripts/feedback.test.mjs scripts/content-read.test.mjs scripts/build-manifest.test.mjs scripts/core-api-call-sites.test.mjs scripts/deal-actions-cli.test.mjs scripts/deal-actions.test.mjs scripts/deal-memory-actions-cli.test.mjs scripts/deal-memory-actions.test.mjs scripts/mcp-four-action-surface.test.mjs scripts/page-schema-cli.test.mjs scripts/page-schema.test.mjs scripts/server-compatibility.test.mjs scripts/verify-core-api-contract.test.mjs && node scripts/verify-core-api-contract.mjs",
|
|
10
10
|
"verify:artifact": "node scripts/verify-release-artifact.mjs",
|
|
11
11
|
"verify:release": "npm test && npm run verify:artifact && node scripts/verify-tarball-clean.mjs",
|
|
12
12
|
"prepack": "node scripts/prepare-build-manifest.mjs",
|