@birdie_moblie/open_spec 2.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +13 -0
- package/.claude-plugin/marketplace.json +12 -0
- package/.claude-plugin/plugin.json +9 -0
- package/.codex-plugin/plugin.json +9 -0
- package/LICENSE +21 -0
- package/README.md +111 -0
- package/bin/openspec.js +3 -0
- package/dist/cli/commands/change.d.ts +3 -0
- package/dist/cli/commands/change.js +37 -0
- package/dist/cli/commands/ext.d.ts +3 -0
- package/dist/cli/commands/ext.js +196 -0
- package/dist/cli/commands/figma.d.ts +3 -0
- package/dist/cli/commands/figma.js +73 -0
- package/dist/cli/commands/init.d.ts +3 -0
- package/dist/cli/commands/init.js +71 -0
- package/dist/cli/commands/opsx.d.ts +3 -0
- package/dist/cli/commands/opsx.js +324 -0
- package/dist/cli/commands/usage.d.ts +3 -0
- package/dist/cli/commands/usage.js +21 -0
- package/dist/cli/output.d.ts +8 -0
- package/dist/cli/output.js +17 -0
- package/dist/cli/program.d.ts +4 -0
- package/dist/cli/program.js +42 -0
- package/dist/cli/register.d.ts +3 -0
- package/dist/cli/register.js +13 -0
- package/dist/cli.d.ts +2 -0
- package/dist/cli.js +3 -0
- package/dist/config/load.d.ts +7 -0
- package/dist/config/load.js +55 -0
- package/dist/config/presets.d.ts +129 -0
- package/dist/config/presets.js +125 -0
- package/dist/config/schema.d.ts +281 -0
- package/dist/config/schema.js +178 -0
- package/dist/core/archive.d.ts +5 -0
- package/dist/core/archive.js +33 -0
- package/dist/core/change.d.ts +12 -0
- package/dist/core/change.js +69 -0
- package/dist/core/code-changes.d.ts +9 -0
- package/dist/core/code-changes.js +31 -0
- package/dist/core/flow.d.ts +6 -0
- package/dist/core/flow.js +99 -0
- package/dist/core/gates.d.ts +5 -0
- package/dist/core/gates.js +71 -0
- package/dist/core/instructions.d.ts +21 -0
- package/dist/core/instructions.js +102 -0
- package/dist/core/source-operations.d.ts +25 -0
- package/dist/core/source-operations.js +94 -0
- package/dist/core/sources.d.ts +33 -0
- package/dist/core/sources.js +234 -0
- package/dist/core/state.d.ts +15 -0
- package/dist/core/state.js +75 -0
- package/dist/core/status.d.ts +31 -0
- package/dist/core/status.js +22 -0
- package/dist/core/tasks.d.ts +15 -0
- package/dist/core/tasks.js +143 -0
- package/dist/core/todo.d.ts +31 -0
- package/dist/core/todo.js +114 -0
- package/dist/core/types.d.ts +297 -0
- package/dist/core/types.js +227 -0
- package/dist/core/verify.d.ts +229 -0
- package/dist/core/verify.js +391 -0
- package/dist/core/worker-evidence.d.ts +71 -0
- package/dist/core/worker-evidence.js +253 -0
- package/dist/core/worker.d.ts +22 -0
- package/dist/core/worker.js +478 -0
- package/dist/delivery/claude-install.d.ts +4 -0
- package/dist/delivery/claude-install.js +44 -0
- package/dist/delivery/distribution.d.ts +3 -0
- package/dist/delivery/distribution.js +53 -0
- package/dist/delivery/skills.d.ts +36 -0
- package/dist/delivery/skills.js +178 -0
- package/dist/figma/cache.d.ts +19 -0
- package/dist/figma/cache.js +115 -0
- package/dist/figma/claude.d.ts +14 -0
- package/dist/figma/claude.js +121 -0
- package/dist/figma/collect.d.ts +3 -0
- package/dist/figma/collect.js +57 -0
- package/dist/figma/index.d.ts +35 -0
- package/dist/figma/index.js +178 -0
- package/dist/figma/media.d.ts +5 -0
- package/dist/figma/media.js +77 -0
- package/dist/figma/response.d.ts +9 -0
- package/dist/figma/response.js +27 -0
- package/dist/hooks/command-context.d.ts +19 -0
- package/dist/hooks/command-context.js +33 -0
- package/dist/hooks/engine.d.ts +61 -0
- package/dist/hooks/engine.js +314 -0
- package/dist/hooks/plugin-runtime.d.ts +188 -0
- package/dist/hooks/plugin-runtime.js +198 -0
- package/dist/index.d.ts +99 -0
- package/dist/index.js +185 -0
- package/dist/marketplace/adapters.d.ts +202 -0
- package/dist/marketplace/adapters.js +203 -0
- package/dist/marketplace/builtin.d.ts +9 -0
- package/dist/marketplace/builtin.js +9 -0
- package/dist/marketplace/install.d.ts +38 -0
- package/dist/marketplace/install.js +108 -0
- package/dist/marketplace/registry.d.ts +71 -0
- package/dist/marketplace/registry.js +342 -0
- package/dist/marketplace/upgrade.d.ts +39 -0
- package/dist/marketplace/upgrade.js +104 -0
- package/dist/shared/command-log.d.ts +9 -0
- package/dist/shared/command-log.js +55 -0
- package/dist/shared/errors.d.ts +7 -0
- package/dist/shared/errors.js +14 -0
- package/dist/shared/exec.d.ts +16 -0
- package/dist/shared/exec.js +86 -0
- package/dist/shared/fs.d.ts +18 -0
- package/dist/shared/fs.js +93 -0
- package/dist/shared/git.d.ts +22 -0
- package/dist/shared/git.js +132 -0
- package/dist/shared/hash.d.ts +5 -0
- package/dist/shared/hash.js +29 -0
- package/dist/shared/paths.d.ts +16 -0
- package/dist/shared/paths.js +97 -0
- package/dist/shared/pkg.d.ts +3 -0
- package/dist/shared/pkg.js +23 -0
- package/dist/shared/version.d.ts +9 -0
- package/dist/shared/version.js +34 -0
- package/dist/usage/analyze.d.ts +5 -0
- package/dist/usage/analyze.js +212 -0
- package/dist/usage/claude.d.ts +9 -0
- package/dist/usage/claude.js +59 -0
- package/dist/usage/collect.d.ts +11 -0
- package/dist/usage/collect.js +71 -0
- package/dist/usage/doctor.d.ts +72 -0
- package/dist/usage/doctor.js +121 -0
- package/dist/usage/parse.d.ts +28 -0
- package/dist/usage/parse.js +345 -0
- package/dist/usage/types.d.ts +103 -0
- package/dist/usage/types.js +2 -0
- package/docs/getting-started.md +20 -0
- package/docs/guides/configuration.md +33 -0
- package/docs/guides/cost-benchmark.md +88 -0
- package/docs/guides/plugin-migration.md +65 -0
- package/docs/guides/smart-figma.md +51 -0
- package/docs/guides/source-reading.md +35 -0
- package/docs/guides/ui-conventions-template.md +39 -0
- package/docs/guides/worker-evidence.md +73 -0
- package/docs/guides/workflow.md +74 -0
- package/docs/maintainers/architecture.md +27 -0
- package/docs/maintainers/plugin-runtime-contract.md +55 -0
- package/package.json +78 -0
- package/plugin.json +9 -0
- package/presets/backend-service.yaml +19 -0
- package/presets/flutter-mobile/plugin-hooks.json +68 -0
- package/presets/flutter-mobile/ui-conventions.md +122 -0
- package/presets/flutter-mobile.yaml +205 -0
- package/presets/web-product.yaml +23 -0
- package/skills/opsx-apply/SKILL.md +35 -0
- package/skills/opsx-apply/nodes/blocked.md +7 -0
- package/skills/opsx-apply/nodes/code-task.md +29 -0
- package/skills/opsx-apply/nodes/gate-a.md +9 -0
- package/skills/opsx-apply/nodes/stop.md +5 -0
- package/skills/opsx-apply/nodes/task-build.md +50 -0
- package/skills/opsx-apply/nodes/task-repair.md +20 -0
- package/skills/opsx-apply/nodes/task-scout.md +28 -0
- package/skills/opsx-apply/nodes/task-verify.md +38 -0
- package/skills/opsx-archive/SKILL.md +24 -0
- package/skills/opsx-archive/nodes/archive.md +15 -0
- package/skills/opsx-explore/SKILL.md +20 -0
- package/skills/opsx-explore/nodes/explore.md +3 -0
- package/skills/opsx-fix-verify/SKILL.md +30 -0
- package/skills/opsx-fix-verify/nodes/repair.md +11 -0
- package/skills/opsx-fix-verify/nodes/reverify.md +3 -0
- package/skills/opsx-fix-verify/nodes/stop.md +3 -0
- package/skills/opsx-gate-decision/SKILL.md +20 -0
- package/skills/opsx-gate-decision/nodes/decide.md +15 -0
- package/skills/opsx-propose/SKILL.md +36 -0
- package/skills/opsx-propose/nodes/blocked.md +13 -0
- package/skills/opsx-propose/nodes/collect.md +35 -0
- package/skills/opsx-propose/nodes/parse.md +72 -0
- package/skills/opsx-propose/nodes/plan.md +65 -0
- package/skills/opsx-propose/nodes/stop.md +17 -0
- package/skills/opsx-propose/nodes/tech.md +77 -0
- package/skills/opsx-propose/scripts/read-json-source.mjs +22 -0
- package/skills/opsx-resume/SKILL.md +21 -0
- package/skills/opsx-resume/nodes/route.md +12 -0
- package/skills/opsx-source-update/SKILL.md +23 -0
- package/skills/opsx-source-update/nodes/update.md +14 -0
- package/skills/opsx-verify/SKILL.md +32 -0
- package/skills/opsx-verify/nodes/blocked.md +3 -0
- package/skills/opsx-verify/nodes/failed.md +3 -0
- package/skills/opsx-verify/nodes/gate-b.md +7 -0
- package/skills/opsx-verify/nodes/stop.md +5 -0
- package/skills/opsx-verify/nodes/verify.md +10 -0
|
@@ -0,0 +1,345 @@
|
|
|
1
|
+
import { sha256 } from '../shared/hash.js';
|
|
2
|
+
export const object = (value) => value && typeof value === 'object' && !Array.isArray(value) ? value : {};
|
|
3
|
+
export const string = (value) => typeof value === 'string' && value ? value : null;
|
|
4
|
+
export const number = (value) => typeof value === 'number' && Number.isSafeInteger(value) && value >= 0 ? value : null;
|
|
5
|
+
export const zeroTokens = () => ({ input: 0, output: 0, cachedInput: 0, cacheWriteInput: 0, reasoningOutput: 0, total: 0 });
|
|
6
|
+
export function sumTokens(values) {
|
|
7
|
+
return values.reduce((a, b) => ({ input: a.input + b.input, output: a.output + b.output, total: a.total + b.total,
|
|
8
|
+
cachedInput: a.cachedInput === null || b.cachedInput === null ? null : a.cachedInput + b.cachedInput,
|
|
9
|
+
cacheWriteInput: a.cacheWriteInput === null || b.cacheWriteInput === null ? null : a.cacheWriteInput + b.cacheWriteInput,
|
|
10
|
+
reasoningOutput: a.reasoningOutput === null || b.reasoningOutput === null ? null : a.reasoningOutput + b.reasoningOutput }), zeroTokens());
|
|
11
|
+
}
|
|
12
|
+
export function mergeClaudeTokens(a, b) {
|
|
13
|
+
const max = (x, y) => x === null ? y : y === null ? x : Math.max(x, y);
|
|
14
|
+
const complete = (value) => value.cachedInput !== null && value.cacheWriteInput !== null;
|
|
15
|
+
if (complete(a) !== complete(b)) {
|
|
16
|
+
// A partial stream can report provisional total input in input_tokens.
|
|
17
|
+
// Prefer the complete breakdown as a unit; do not add its cache to that total.
|
|
18
|
+
const chosen = complete(a) ? a : b;
|
|
19
|
+
const output = Math.max(a.output, b.output);
|
|
20
|
+
return { ...chosen, output, total: chosen.input + output };
|
|
21
|
+
}
|
|
22
|
+
const cachedInput = max(a.cachedInput, b.cachedInput);
|
|
23
|
+
const cacheWriteInput = max(a.cacheWriteInput, b.cacheWriteInput);
|
|
24
|
+
const input = Math.max(a.input - (a.cachedInput ?? 0) - (a.cacheWriteInput ?? 0), b.input - (b.cachedInput ?? 0) - (b.cacheWriteInput ?? 0)) + (cachedInput ?? 0) + (cacheWriteInput ?? 0);
|
|
25
|
+
const output = Math.max(a.output, b.output);
|
|
26
|
+
return { input, output, total: input + output, cachedInput, cacheWriteInput, reasoningOutput: max(a.reasoningOutput, b.reasoningOutput) };
|
|
27
|
+
}
|
|
28
|
+
export function parseUsageStream(file, text) {
|
|
29
|
+
const stream = { threadId: `unknown-${sha256(file).slice(0, 16)}`, parentThreadId: null, provider: 'codex', source: 'exec', file,
|
|
30
|
+
slices: [], issues: [], tools: new Set(), spawns: new Set(), epochs: 1, startedAt: null, endedAt: null, collabObserved: false, apiCalls: null, toolMessages: new Map() };
|
|
31
|
+
let model = null;
|
|
32
|
+
let effort = null;
|
|
33
|
+
let workerId = null;
|
|
34
|
+
let stepId = null;
|
|
35
|
+
let previous = null;
|
|
36
|
+
let terminal = false;
|
|
37
|
+
let execTurn = 0;
|
|
38
|
+
const completedTurns = new Map();
|
|
39
|
+
const seenSnapshots = new Set();
|
|
40
|
+
const claude = new Map();
|
|
41
|
+
const requests = new Map();
|
|
42
|
+
const requestCheckpoints = new Map();
|
|
43
|
+
const compactions = new Set();
|
|
44
|
+
let requestCheckpoint = null;
|
|
45
|
+
const issue = (code, line, severity = 'incomplete') => stream.issues.push({ code, severity, file, line });
|
|
46
|
+
const attribution = (value) => {
|
|
47
|
+
const source = typeof value === 'string' ? value : JSON.stringify(value) ?? '';
|
|
48
|
+
if (/worker\s+(accept|cancel)\b/.test(source)) {
|
|
49
|
+
workerId = null;
|
|
50
|
+
stepId = null;
|
|
51
|
+
return;
|
|
52
|
+
}
|
|
53
|
+
const packet = source.match(/\/workers\/(([A-Za-z0-9_.-]+)-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})\.packet\.json/);
|
|
54
|
+
if (packet) {
|
|
55
|
+
workerId = packet[1];
|
|
56
|
+
stepId = packet[2];
|
|
57
|
+
}
|
|
58
|
+
else {
|
|
59
|
+
const step = source.match(/worker\s+create\b[^\n]*?--step\s+["']?([A-Za-z0-9_.-]+)/);
|
|
60
|
+
if (step) {
|
|
61
|
+
stepId = step[1];
|
|
62
|
+
workerId = null;
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
};
|
|
66
|
+
const snapshot = (raw, line, timestamp) => {
|
|
67
|
+
const current = normalizeCodex(raw);
|
|
68
|
+
if (!current) {
|
|
69
|
+
issue('invalid_usage', line);
|
|
70
|
+
return;
|
|
71
|
+
}
|
|
72
|
+
if (number(raw.total_tokens) !== null && raw.total_tokens !== current.total)
|
|
73
|
+
issue('total_tokens_mismatch', line);
|
|
74
|
+
if (current.cachedInput !== null && current.cachedInput > current.input)
|
|
75
|
+
issue('cache_exceeds_input', line);
|
|
76
|
+
if (current.reasoningOutput !== null && current.reasoningOutput > current.output)
|
|
77
|
+
issue('reasoning_exceeds_output', line);
|
|
78
|
+
const key = JSON.stringify(current);
|
|
79
|
+
if (seenSnapshots.has(key)) {
|
|
80
|
+
if (previous && JSON.stringify(previous) !== key)
|
|
81
|
+
issue('ambiguous_snapshot_replay', line);
|
|
82
|
+
return;
|
|
83
|
+
}
|
|
84
|
+
if (previous && (current.input < previous.input || current.output < previous.output)) {
|
|
85
|
+
previous = null;
|
|
86
|
+
seenSnapshots.clear();
|
|
87
|
+
stream.epochs += 1;
|
|
88
|
+
issue('counter_reset', line, 'warning');
|
|
89
|
+
}
|
|
90
|
+
seenSnapshots.add(key);
|
|
91
|
+
const delta = previous ? subtractTokens(current, previous) : current;
|
|
92
|
+
previous = current;
|
|
93
|
+
if (delta.input === 0 && delta.output === 0)
|
|
94
|
+
return;
|
|
95
|
+
stream.slices.push({ key: `${stream.threadId}:${stream.epochs}:${line}`, threadId: stream.threadId, model, effort, workerId, stepId, timestamp, tokens: delta, file, line });
|
|
96
|
+
};
|
|
97
|
+
text.split('\n').forEach((lineText, index) => {
|
|
98
|
+
if (!lineText.trim())
|
|
99
|
+
return;
|
|
100
|
+
const line = index + 1;
|
|
101
|
+
let row;
|
|
102
|
+
try {
|
|
103
|
+
row = object(JSON.parse(lineText));
|
|
104
|
+
}
|
|
105
|
+
catch {
|
|
106
|
+
issue('malformed_jsonl', line);
|
|
107
|
+
return;
|
|
108
|
+
}
|
|
109
|
+
const timestamp = string(row.timestamp);
|
|
110
|
+
if (timestamp) {
|
|
111
|
+
stream.startedAt ??= timestamp;
|
|
112
|
+
stream.endedAt = timestamp;
|
|
113
|
+
}
|
|
114
|
+
const payload = object(row.payload);
|
|
115
|
+
if (row.type === 'session_meta') {
|
|
116
|
+
stream.source = 'rollout';
|
|
117
|
+
stream.threadId = string(payload.id) ?? stream.threadId;
|
|
118
|
+
const source = object(payload.source);
|
|
119
|
+
const subagent = object(source.subagent);
|
|
120
|
+
stream.parentThreadId = string(object(subagent.thread_spawn).parent_thread_id) ?? string(object(subagent.spawn).parent_thread_id);
|
|
121
|
+
}
|
|
122
|
+
if (row.type === 'thread.started')
|
|
123
|
+
stream.threadId = string(row.thread_id) ?? stream.threadId;
|
|
124
|
+
if (row.type === 'turn.started') {
|
|
125
|
+
terminal = false;
|
|
126
|
+
execTurn += 1;
|
|
127
|
+
}
|
|
128
|
+
if (row.type === 'event_msg' && payload.type === 'task_started')
|
|
129
|
+
terminal = false;
|
|
130
|
+
if (row.type === 'turn.completed' || (row.type === 'event_msg' && payload.type === 'task_complete'))
|
|
131
|
+
terminal = true;
|
|
132
|
+
if (row.type === 'turn_context') {
|
|
133
|
+
model = string(payload.model);
|
|
134
|
+
effort = string(payload.effort) ?? string(payload.reasoning_effort);
|
|
135
|
+
}
|
|
136
|
+
if (row.type === 'compacted') {
|
|
137
|
+
const id = string(payload.compaction_response_id);
|
|
138
|
+
if (id)
|
|
139
|
+
compactions.add(id);
|
|
140
|
+
}
|
|
141
|
+
if (row.type === 'token_usage_record') {
|
|
142
|
+
const id = string(payload.response_id);
|
|
143
|
+
const tokens = normalizeCodex(object(payload.usage));
|
|
144
|
+
if (!id || !tokens || payload.thread_id !== stream.threadId)
|
|
145
|
+
issue('invalid_request_usage', line);
|
|
146
|
+
else {
|
|
147
|
+
const checkpoint = normalizeCodex(object(payload.thread_token_usage));
|
|
148
|
+
const previousCheckpoint = requestCheckpoints.get(id);
|
|
149
|
+
if (checkpoint && previousCheckpoint) {
|
|
150
|
+
if (!matchingTokens(checkpoint, previousCheckpoint))
|
|
151
|
+
issue('conflicting_request_checkpoint', line);
|
|
152
|
+
for (const key of ['cachedInput', 'cacheWriteInput', 'reasoningOutput'])
|
|
153
|
+
previousCheckpoint[key] ??= checkpoint[key];
|
|
154
|
+
}
|
|
155
|
+
else if (checkpoint)
|
|
156
|
+
requestCheckpoints.set(id, checkpoint);
|
|
157
|
+
const existing = requests.get(id);
|
|
158
|
+
if (existing && JSON.stringify(existing.tokens) !== JSON.stringify(tokens))
|
|
159
|
+
issue('conflicting_request_usage', line);
|
|
160
|
+
else if (!existing) {
|
|
161
|
+
requests.set(id, { key: id, threadId: stream.threadId, model, effort, workerId, stepId, timestamp, tokens, file, line });
|
|
162
|
+
requestCheckpoint = checkpoint;
|
|
163
|
+
if (tokens.cachedInput !== null && tokens.cachedInput > tokens.input)
|
|
164
|
+
issue('cache_exceeds_input', line);
|
|
165
|
+
if (tokens.reasoningOutput !== null && tokens.reasoningOutput > tokens.output)
|
|
166
|
+
issue('reasoning_exceeds_output', line);
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
}
|
|
170
|
+
if (row.type === 'response_item') {
|
|
171
|
+
if (payload.type === 'function_call' || payload.type === 'custom_tool_call') {
|
|
172
|
+
const id = string(payload.call_id) ?? `${file}:${line}`;
|
|
173
|
+
stream.tools.add(id);
|
|
174
|
+
attribution(payload.arguments ?? payload.input ?? '');
|
|
175
|
+
if (string(payload.name)?.endsWith('spawn_agent'))
|
|
176
|
+
stream.spawns.add(id);
|
|
177
|
+
}
|
|
178
|
+
if (payload.type === 'message' && payload.role === 'user')
|
|
179
|
+
attribution(payload.content);
|
|
180
|
+
}
|
|
181
|
+
if (row.type === 'event_msg' && payload.type === 'token_count' && object(payload.info).total_token_usage)
|
|
182
|
+
snapshot(object(object(payload.info).total_token_usage), line, timestamp);
|
|
183
|
+
if (row.type === 'turn.completed' && row.usage) {
|
|
184
|
+
// exec usage is a turn increment, unlike rollout total_token_usage snapshots.
|
|
185
|
+
const tokens = normalizeCodex(object(row.usage));
|
|
186
|
+
const key = JSON.stringify(tokens);
|
|
187
|
+
if (!tokens)
|
|
188
|
+
issue('invalid_usage', line);
|
|
189
|
+
else if (completedTurns.has(execTurn)) {
|
|
190
|
+
if (completedTurns.get(execTurn) !== key)
|
|
191
|
+
issue('conflicting_turn_usage', line);
|
|
192
|
+
}
|
|
193
|
+
else {
|
|
194
|
+
completedTurns.set(execTurn, key);
|
|
195
|
+
if (tokens.cachedInput !== null && tokens.cachedInput > tokens.input)
|
|
196
|
+
issue('cache_exceeds_input', line);
|
|
197
|
+
if (tokens.reasoningOutput !== null && tokens.reasoningOutput > tokens.output)
|
|
198
|
+
issue('reasoning_exceeds_output', line);
|
|
199
|
+
stream.slices.push({ key: `${stream.threadId}:exec-turn:${execTurn}`, threadId: stream.threadId, model, effort, workerId, stepId, timestamp, tokens, file, line });
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
if (row.type === 'turn.failed') {
|
|
203
|
+
if (row.usage)
|
|
204
|
+
snapshot(object(row.usage), line, timestamp);
|
|
205
|
+
issue(row.usage ? 'failed_turn' : 'failed_turn_usage_unverified', line, row.usage ? 'warning' : 'incomplete');
|
|
206
|
+
}
|
|
207
|
+
if (row.method === 'model/rerouted' || payload.type === 'model_rerouted') {
|
|
208
|
+
model = string(object(row.params).toModel) ?? string(payload.to_model);
|
|
209
|
+
issue('model_rerouted', line);
|
|
210
|
+
}
|
|
211
|
+
const item = object(row.item);
|
|
212
|
+
if (row.type === 'item.completed' && ['command_execution', 'mcp_tool_call'].includes(String(item.type))) {
|
|
213
|
+
stream.tools.add(string(item.id) ?? `${file}:${line}`);
|
|
214
|
+
attribution(item.command ?? item.arguments ?? '');
|
|
215
|
+
}
|
|
216
|
+
if (String(item.type).includes('collab'))
|
|
217
|
+
stream.collabObserved = true;
|
|
218
|
+
const message = object(row.message);
|
|
219
|
+
if (row.type === 'assistant' && message.usage) {
|
|
220
|
+
stream.provider = 'claude';
|
|
221
|
+
stream.source = 'claude';
|
|
222
|
+
if (message.model === '<synthetic>') {
|
|
223
|
+
// Local interruption/compaction markers are not model calls. An API error
|
|
224
|
+
// with synthetic zero usage does not prove the failed request was free.
|
|
225
|
+
if (row.isApiErrorMessage === true)
|
|
226
|
+
issue('claude_failed_request_usage_unverified', line);
|
|
227
|
+
else
|
|
228
|
+
issue('claude_synthetic_message', line, 'warning');
|
|
229
|
+
return;
|
|
230
|
+
}
|
|
231
|
+
const session = string(row.sessionId) ?? stream.threadId;
|
|
232
|
+
const agent = string(row.agentId) ?? (file.includes('/subagents/') ? file.match(/agent-([^/]+)\.jsonl$/)?.[1] ?? null : null);
|
|
233
|
+
stream.threadId = agent ? `${session}::agent:${agent}` : session;
|
|
234
|
+
stream.parentThreadId = string(row.parentSessionId) ?? (agent ? session : null);
|
|
235
|
+
model = string(message.model) ?? model;
|
|
236
|
+
effort = string(row.effort) ?? string(row.reasoning_effort) ?? effort;
|
|
237
|
+
const id = string(message.id);
|
|
238
|
+
if (!id) {
|
|
239
|
+
issue('claude_message_id_missing', line);
|
|
240
|
+
return;
|
|
241
|
+
}
|
|
242
|
+
const usage = object(message.usage);
|
|
243
|
+
const previous = claude.get(id);
|
|
244
|
+
const raw = { ...(previous?.raw ?? {}) };
|
|
245
|
+
const inputKeys = ['input_tokens', 'cache_read_input_tokens', 'cache_creation_input_tokens'];
|
|
246
|
+
const completeInput = (value) => inputKeys.every((key) => number(value[key]) !== null);
|
|
247
|
+
if (completeInput(usage)) {
|
|
248
|
+
for (const key of inputKeys)
|
|
249
|
+
raw[key] = usage[key];
|
|
250
|
+
}
|
|
251
|
+
else if (!completeInput(raw)) {
|
|
252
|
+
// Genuine partial deltas may omit unchanged fields. Preserve them until
|
|
253
|
+
// a complete tuple arrives, including an explicit downward correction.
|
|
254
|
+
for (const key of inputKeys)
|
|
255
|
+
if (number(usage[key]) !== null)
|
|
256
|
+
raw[key] = usage[key];
|
|
257
|
+
}
|
|
258
|
+
const output = number(usage.output_tokens);
|
|
259
|
+
if (output !== null)
|
|
260
|
+
raw.output_tokens = Math.max(number(raw.output_tokens) ?? 0, output);
|
|
261
|
+
claude.set(id, { raw, slice: previous?.slice ?? { key: id, threadId: stream.threadId, model, effort, workerId, stepId, timestamp, file, line } });
|
|
262
|
+
for (const block of Array.isArray(message.content) ? message.content : []) {
|
|
263
|
+
const tool = object(block);
|
|
264
|
+
if (tool.type !== 'tool_use')
|
|
265
|
+
continue;
|
|
266
|
+
const toolId = string(tool.id) ?? `${file}:${line}`;
|
|
267
|
+
stream.tools.add(toolId);
|
|
268
|
+
stream.toolMessages.set(toolId, id);
|
|
269
|
+
attribution(tool.input);
|
|
270
|
+
if (['Agent', 'Task'].includes(String(tool.name)))
|
|
271
|
+
stream.spawns.add(toolId);
|
|
272
|
+
}
|
|
273
|
+
}
|
|
274
|
+
});
|
|
275
|
+
if (stream.provider === 'claude') {
|
|
276
|
+
stream.slices = [];
|
|
277
|
+
for (const { raw, slice } of claude.values()) {
|
|
278
|
+
const input = number(raw.input_tokens);
|
|
279
|
+
const output = number(raw.output_tokens);
|
|
280
|
+
if (input === null || output === null) {
|
|
281
|
+
issue('invalid_usage', slice.line);
|
|
282
|
+
continue;
|
|
283
|
+
}
|
|
284
|
+
const cachedInput = number(raw.cache_read_input_tokens);
|
|
285
|
+
const cacheWriteInput = number(raw.cache_creation_input_tokens);
|
|
286
|
+
if (cachedInput === null || cacheWriteInput === null)
|
|
287
|
+
issue('claude_cache_fields_missing', slice.line);
|
|
288
|
+
const totalInput = input + (cachedInput ?? 0) + (cacheWriteInput ?? 0);
|
|
289
|
+
stream.slices.push({ ...slice, tokens: { input: totalInput, output, cachedInput, cacheWriteInput, reasoningOutput: null, total: totalInput + output } });
|
|
290
|
+
}
|
|
291
|
+
}
|
|
292
|
+
if (stream.threadId.startsWith('unknown-'))
|
|
293
|
+
stream.issues.push({ code: 'thread_id_missing', severity: 'incomplete', file });
|
|
294
|
+
if (stream.provider === 'codex' && !terminal)
|
|
295
|
+
stream.issues.push({ code: 'session_completion_unverified', severity: 'incomplete', file });
|
|
296
|
+
if (requests.size) {
|
|
297
|
+
const recorded = sumTokens([...requests.values()].map((slice) => slice.tokens));
|
|
298
|
+
const snapshots = sumTokens(stream.slices.map((slice) => slice.tokens));
|
|
299
|
+
const compactionSlices = [...compactions].flatMap((id) => requests.has(id) ? [requests.get(id)] : []);
|
|
300
|
+
// Some Codex versions exclude automatic compaction from legacy token_count
|
|
301
|
+
// snapshots, while response records and their thread checkpoint include it.
|
|
302
|
+
// Reconcile only an explicitly identified, fully accounted-for difference.
|
|
303
|
+
const reconciledCompaction = compactionSlices.length > 0 && compactionSlices.length === compactions.size
|
|
304
|
+
&& requestCheckpoint !== null && matchingTokens(recorded, requestCheckpoint)
|
|
305
|
+
&& matchingTokens(recorded, sumTokens([snapshots, ...compactionSlices.map((slice) => slice.tokens)]));
|
|
306
|
+
if (!stream.slices.length || (recorded.input === snapshots.input && recorded.output === snapshots.output) || reconciledCompaction) {
|
|
307
|
+
stream.slices = [...requests.values()];
|
|
308
|
+
stream.apiCalls = requests.size;
|
|
309
|
+
for (const slice of compactionSlices) {
|
|
310
|
+
slice.workerId = null;
|
|
311
|
+
slice.stepId = 'compaction';
|
|
312
|
+
}
|
|
313
|
+
if (reconciledCompaction)
|
|
314
|
+
stream.issues.push({ code: 'compaction_usage_reconciled', severity: 'warning', file });
|
|
315
|
+
}
|
|
316
|
+
else {
|
|
317
|
+
stream.issues.push({ code: 'request_snapshot_mismatch', severity: 'incomplete', file });
|
|
318
|
+
const input = Math.max(0, recorded.input - snapshots.input);
|
|
319
|
+
const output = Math.max(0, recorded.output - snapshots.output);
|
|
320
|
+
if (input || output)
|
|
321
|
+
stream.slices.push({ key: `${stream.threadId}:request-gap`, threadId: stream.threadId, model: null, effort: null,
|
|
322
|
+
workerId: null, stepId: null, timestamp: null, file, line: 1, tokens: { input, output, total: input + output, cachedInput: null, cacheWriteInput: null, reasoningOutput: null } });
|
|
323
|
+
}
|
|
324
|
+
}
|
|
325
|
+
for (const slice of stream.slices)
|
|
326
|
+
slice.threadId = stream.threadId;
|
|
327
|
+
return stream;
|
|
328
|
+
}
|
|
329
|
+
function matchingTokens(a, b) {
|
|
330
|
+
return a.input === b.input && a.output === b.output
|
|
331
|
+
&& ['cachedInput', 'cacheWriteInput', 'reasoningOutput'].every((key) => a[key] === null || b[key] === null || a[key] === b[key]);
|
|
332
|
+
}
|
|
333
|
+
function normalizeCodex(raw) {
|
|
334
|
+
const input = number(raw.input_tokens);
|
|
335
|
+
const output = number(raw.output_tokens);
|
|
336
|
+
if (input === null || output === null)
|
|
337
|
+
return null;
|
|
338
|
+
return { input, output, cachedInput: number(raw.cached_input_tokens), cacheWriteInput: number(raw.cache_write_input_tokens), reasoningOutput: number(raw.reasoning_output_tokens), total: input + output };
|
|
339
|
+
}
|
|
340
|
+
function subtractTokens(current, previous) {
|
|
341
|
+
const subtract = (a, b) => a === null || b === null || a < b ? null : a - b;
|
|
342
|
+
return { input: current.input - previous.input, output: current.output - previous.output, total: current.total - previous.total,
|
|
343
|
+
cachedInput: subtract(current.cachedInput, previous.cachedInput), cacheWriteInput: subtract(current.cacheWriteInput, previous.cacheWriteInput), reasoningOutput: subtract(current.reasoningOutput, previous.reasoningOutput) };
|
|
344
|
+
}
|
|
345
|
+
//# sourceMappingURL=parse.js.map
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
export interface Tokens {
|
|
2
|
+
input: number;
|
|
3
|
+
output: number;
|
|
4
|
+
cachedInput: number | null;
|
|
5
|
+
cacheWriteInput: number | null;
|
|
6
|
+
reasoningOutput: number | null;
|
|
7
|
+
total: number;
|
|
8
|
+
}
|
|
9
|
+
export interface UsageIssue {
|
|
10
|
+
code: string;
|
|
11
|
+
severity: 'warning' | 'incomplete';
|
|
12
|
+
file: string;
|
|
13
|
+
line?: number;
|
|
14
|
+
threadId?: string;
|
|
15
|
+
}
|
|
16
|
+
export interface UsageSlice {
|
|
17
|
+
key: string;
|
|
18
|
+
threadId: string;
|
|
19
|
+
model: string | null;
|
|
20
|
+
effort: string | null;
|
|
21
|
+
workerId: string | null;
|
|
22
|
+
stepId: string | null;
|
|
23
|
+
timestamp: string | null;
|
|
24
|
+
tokens: Tokens;
|
|
25
|
+
file: string;
|
|
26
|
+
line: number;
|
|
27
|
+
attribution?: 'event_window' | 'log_hint' | 'request_id' | 'unattributed';
|
|
28
|
+
}
|
|
29
|
+
export interface SessionUsage {
|
|
30
|
+
threadId: string;
|
|
31
|
+
parentThreadId: string | null;
|
|
32
|
+
provider: 'codex' | 'claude';
|
|
33
|
+
source: 'rollout' | 'exec' | 'claude';
|
|
34
|
+
files: string[];
|
|
35
|
+
models: string[];
|
|
36
|
+
efforts: string[];
|
|
37
|
+
tokens: Tokens;
|
|
38
|
+
usageObservations: number;
|
|
39
|
+
toolCalls: number;
|
|
40
|
+
spawnCalls: number;
|
|
41
|
+
epochs: number;
|
|
42
|
+
apiCalls: number | null;
|
|
43
|
+
startedAt: string | null;
|
|
44
|
+
endedAt: string | null;
|
|
45
|
+
elapsedMs: number | null;
|
|
46
|
+
}
|
|
47
|
+
export interface UsageReport {
|
|
48
|
+
version: 1;
|
|
49
|
+
changeName: string;
|
|
50
|
+
totals: Tokens;
|
|
51
|
+
sessions: SessionUsage[];
|
|
52
|
+
slices: UsageSlice[];
|
|
53
|
+
byStage: Array<{
|
|
54
|
+
stepId: string;
|
|
55
|
+
tokens: Tokens;
|
|
56
|
+
observations: number;
|
|
57
|
+
}>;
|
|
58
|
+
coverage: {
|
|
59
|
+
tokenComplete: boolean;
|
|
60
|
+
modelsKnown: boolean;
|
|
61
|
+
effortsKnown: boolean;
|
|
62
|
+
comparisonEligible: boolean;
|
|
63
|
+
issues: UsageIssue[];
|
|
64
|
+
};
|
|
65
|
+
cost: {
|
|
66
|
+
kind: 'actual' | 'estimated' | 'unavailable';
|
|
67
|
+
usd: number | null;
|
|
68
|
+
source: string | null;
|
|
69
|
+
complete: boolean;
|
|
70
|
+
};
|
|
71
|
+
counts: {
|
|
72
|
+
sessions: number;
|
|
73
|
+
usageObservations: number;
|
|
74
|
+
toolCalls: number;
|
|
75
|
+
apiCalls: number | null;
|
|
76
|
+
};
|
|
77
|
+
}
|
|
78
|
+
export interface UsageOptions {
|
|
79
|
+
changeName: string;
|
|
80
|
+
sessions: string[];
|
|
81
|
+
events?: string;
|
|
82
|
+
expectedModel?: string;
|
|
83
|
+
expectedEffort?: string;
|
|
84
|
+
rates?: RateCard;
|
|
85
|
+
billing?: Billing;
|
|
86
|
+
}
|
|
87
|
+
export interface RateCard {
|
|
88
|
+
source: string;
|
|
89
|
+
models: Record<string, {
|
|
90
|
+
inputPerMillion: number;
|
|
91
|
+
cachedInputPerMillion: number;
|
|
92
|
+
cacheWriteInputPerMillion?: number;
|
|
93
|
+
outputPerMillion: number;
|
|
94
|
+
}>;
|
|
95
|
+
}
|
|
96
|
+
export interface Billing {
|
|
97
|
+
source: string;
|
|
98
|
+
charges: Array<{
|
|
99
|
+
threadId: string;
|
|
100
|
+
usd: number;
|
|
101
|
+
}>;
|
|
102
|
+
}
|
|
103
|
+
//# sourceMappingURL=types.d.ts.map
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# 快速开始
|
|
2
|
+
|
|
3
|
+
1. 安装 Node.js 22 或 24。
|
|
4
|
+
2. `openspec init --tools claude,codex,cursor --yes`,或先用 `--skip-plugins` 只建立项目配置。
|
|
5
|
+
3. 在宿主中调用 `/opsx-propose` 描述目标。
|
|
6
|
+
4. 审查 Gate-A 后调用 `/opsx-apply`。
|
|
7
|
+
5. `/opsx-verify` 通过 Gate-C 后 `/opsx-archive`。
|
|
8
|
+
|
|
9
|
+
`init --preset flutter-mobile --yes` 会按项目配置中的完整 Git SHA(或 `refs/tags/<tag>`)解析 marketplace,校验插件后调用 Claude/Codex 原生 marketplace 与 selector。项目 `openspec/marketplace-lock.json` 记录 resolved SHA、marketplace digest 和 plugin digest;adapter 合同来自插件 `plugin.json` 的 `extensions["org.openspec"]`,不写进 CLI。
|
|
10
|
+
|
|
11
|
+
Claude 的 marketplace 与插件一律以 `--scope local` 安装,写入 `.claude/settings.local.json`(项目级、仅本人、不入库):其中的 marketplace 路径是本机缓存的绝对路径,用 project scope 会随 `.claude/settings.json` 提交给团队成员。每位成员在自己机器上执行一次 `openspec init --yes` 即可;团队共享的是 `openspec/config.yaml` 与 `marketplace-lock.json`。
|
|
12
|
+
|
|
13
|
+
初始化结果可能是:
|
|
14
|
+
|
|
15
|
+
- `complete`:自动步骤全部完成。
|
|
16
|
+
- `pending_user_action`:非 TTY 未确认,或 Cursor 需要在 Customize → Plugins 中完成;退出码 2。
|
|
17
|
+
- `partial`:部分 marketplace/宿主操作失败;退出码 1,并聚合返回 `issues`。
|
|
18
|
+
- `skipped`:显式使用 `--skip-plugins`,跳过清单写入 `openspec/.runtime/init.json`。
|
|
19
|
+
|
|
20
|
+
测试和并行项目可设置 `OPENSPEC_HOME=/path/to/isolated/home`,避免读写真实用户 cache。`node openspec/install-cli.mjs --check` 校验 Node/npm 与项目钉住的精确 CLI 版本,`openspec doctor --json` 同时校验 installer metadata、宿主交付 digest,以及 config 声明的 marketplace/plugin lock 完整性。
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# 项目配置
|
|
2
|
+
|
|
3
|
+
`openspec/config.yaml` 核心字段:
|
|
4
|
+
|
|
5
|
+
- `schema`: 固定 `spec-product`
|
|
6
|
+
- `taskCategories`: 覆盖或扩展默认分类 `ui` / `logic` / `api-codegen` / `test` / `infra`,每类可写 `notes`;设为 `null` 会删除对应默认分类。
|
|
7
|
+
- `hooks`: 统一外挂,`stage` 严格为 `collect` | `parse` | `tech` | `plan` | `execute` | `verify`,`type` 严格为 `notes` | `skill` | `file` | `check` | `command`,并按 stage 校验可用组合与必填字段。`parse` / `tech` / `plan` 支持五种类型,collect 保留 adapter/skill,verify 支持 command/skill/file/check。
|
|
8
|
+
- `verify.reviewScope` / `verify.failurePolicy`
|
|
9
|
+
- `plugins` / `marketplaces`:宿主的 marketplace ref 与 `openspec/marketplace-lock.json` 是实际使用版本的权威记录,可在兼容范围内独立于 preset 升级;`plugins[].minVersion` 可选,声明最低插件版本,见 [插件能力迁移](plugin-migration.md#兼容检查)
|
|
10
|
+
|
|
11
|
+
每个 plan task 必须给出当前有效的 `category`。Task 只接受软提示 `refScope`;`files` / `allowedDirs` 已移除,`refScope` 不构成写入沙箱。
|
|
12
|
+
|
|
13
|
+
parse / tech / plan Worker packet 也携带 `initContext`:`rules` 为空,`hooks` 是该 stage 全部 hook 的 descriptor(notes 也算),回执规则与 code Worker 相同。这是把项目级约定(例如 `file` hook 指向的 UI 约定清单、spec 结构检查)送达 spec 与计划作者的唯一通道;required `file` 缺失时 Worker 应返回 `blocked`。plan packet 的 `planNotes` 只列各分类说明。
|
|
14
|
+
|
|
15
|
+
code Worker packet 的 `initContext` 顺序固定:category notes、配置声明顺序中的匹配 execute Hook;task notes 仅在 `packet.task.notes`,不重复注入。Hook 未写 `taskCategories` 时匹配所有 task;空数组不匹配;包含当前 category 时匹配。五种 Hook 只传 descriptor:skill/file 不内联正文,command 由 Worker 调用 `openspec worker check` 执行并取得 CLI 回执;文字 passed 不能代替执行。
|
|
16
|
+
|
|
17
|
+
execute Hook 的 `required` 默认 `true`。Worker 的 completed result 必须在 `executeHookResults` 中把每个 required Hook 报为 `passed`;未知 ID、重复 ID、缺失、failed 或 skipped 都会被拒收。optional Hook 可 failed/skipped 并写入事件。required skill/file 不可用时 Worker 应返回 `blocked`。
|
|
18
|
+
|
|
19
|
+
verify `command` 由引擎直接启动子进程执行,不经 shell;需要管道或取反时可写 `sh -c '<script>' sh {files}`(`{files}` 必须是独立参数,`splitCommand` 支持单引号)。`command` 作用于**整个变更文件**而非变更行,所以只适合既有代码里不会出现合法命中的规则(如 analyze、test);文案、颜色这类既有代码中大量合法存在的模式不要写成 grep,否则任何触及老文件的 task 都会误报,应交给 `check` 回执与审查类 `skill`。
|
|
20
|
+
|
|
21
|
+
verify 每次创建独立 run。命令、skill/file/check evidence 和 QA report 都必须属于当前 run;required evidence 缺失、file digest 变化或旧 report 不会通过。`reviewScope` 先裁剪引擎 changed-set,`failurePolicy` 只控制 optional 失败,不能放行 required Hook。
|
|
22
|
+
|
|
23
|
+
Config 为严格 schema:重复 Hook ID、重复 collect capability、未知 category 引用、旧 `context/rules`、可变 marketplace branch ref 都会报错。Preset 按 Hook ID 与 plugin selector 幂等合并,同一插件取较高的 `minVersion`;宿主已声明的 marketplace 保留自身的 repo 与 ref,preset 只补充宿主缺少的 marketplace。Preset 还可声明 `scaffold: [{ path, template, description }]`:`init --preset` 与 `preset apply` 会把 `presets/<template>` 复制到项目 `path`,只创建不覆盖;模板首行的 `<!-- openspec:fill-me -->` 标记未删除前,指向该文件的 `file` hook 不能回执 `passed`。
|
|
24
|
+
|
|
25
|
+
`designRefs` 是 task-plan 的可选节点 id 数组,省略/空数组合法,非空项必须是非空字符串。`packet.refs.design` 提供设计来源 locator 与采集 artifact;详细格式与 status 提示见 [工作流](workflow.md#设计节点引用)。
|
|
26
|
+
|
|
27
|
+
`verify` 的 optional `check` 若标 `category: todo`,始终仅提供信息,不被 failurePolicy 提升为阻塞;清单由 `openspec todo` 和 report.pendingWork 提供。其他 check 的原有 failurePolicy 不变。命令全量日志与 2KB 摘要见 [工作流](workflow.md#产物与待办)。
|
|
28
|
+
|
|
29
|
+
插件 Hook 通过 `use: entry@marketplace` 引用根 manifest 的 `extensions.org.openspec.hooks`,首轮支持 skill/command;与 name/command/path 等内联实现互斥。`operation: preview` 切换为 source adapter 的预演入口。宿主 input 为 JSON 参数,inputFiles 为额外依赖文件路径;动态 --input 的顶层值优先。类型、阶段、重复 ID、损坏和缺失入口明确失败。详情及示例见[插件运行契约](../maintainers/plugin-runtime-contract.md)。
|
|
30
|
+
|
|
31
|
+
已定制的宿主迁移使用精确差异,保留 Hook 的类型、command、required、参数、范围和顺序;同 ID 的 preset 合并是整项替换,不能用它覆盖宿主验收命令。
|
|
32
|
+
|
|
33
|
+
parse/tech/plan 的 Hook 可用 capability 限定仅在当前来源账本包含该能力时适用;省略则应用于整个阶段。required tech command 只在适用时禁止 skipTech。
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# 用量分析与成本实验
|
|
2
|
+
|
|
3
|
+
`tools/experiments/` 仅在源码仓库提供,不随 npm 或原生插件运行制品安装。业务夹具和专用工具由独立实验工程提供。
|
|
4
|
+
|
|
5
|
+
## 独立 Claude 工作流验证入口
|
|
6
|
+
|
|
7
|
+
`node tools/experiments/claude-propose-benchmark.mjs <config.json>` 是独立实验入口,只在获授权的有限批次中使用,不属于正常 Worker 调度。配置的 `model`、`effort` 冻结本次请求;必须显式给出,省略即拒绝启动。切换供应商后先核实当前配置,双方使用同一请求模型和 effort 重跑,不拿跨供应商历史运行计算收益。服务端返回名称与实际账单须分别记录;模型名不证明底层模型身份。
|
|
8
|
+
|
|
9
|
+
已完成的 R0 可以作为后续 R1 修复轮次的固定参照,不必每轮付费重跑。复用前核对供应商/账户、请求模型与 effort、Claude/来源工具版本、业务与规则快照、入口提示、来源内容及统计口径;隔离路径可规范化后比较。纯实验记录器修复须独立证明不改变转发内容与权限。记录被复用运行的路径和摘要,保留其原始失败与质量缺口;候选仍按事实清单验收,不能以基线漏采为标准。
|
|
10
|
+
|
|
11
|
+
模型、供应商、来源或其他影响执行的共同条件变化时,旧 R0 只作历史参考;需要收益判断则在匹配条件下建立新基线。实时来源在候选返回后再核对摘要,内容不同不计算优化收益。复用单次历史基线只能形成单轮比较,不能排除服务负载与随机性,也不能宣称稳定提速或实际账单节省。
|
|
12
|
+
|
|
13
|
+
工作流 fixture 必须包含由被测包 `generateInstallCli` 生成、版本匹配的 `openspec/install-cli.mjs`,并将该文件纳入 `inputs` 摘要。这里只生成并核对文件,不执行安装,不覆盖项目配置。冻结业务输入、提示和转发脚本之后再启动;缺文件、版本不符或提示违反场景声明的歧义检查时,在调用 Claude 前失败。
|
|
14
|
+
|
|
15
|
+
提示中的工具示例取自所选插件的真实命令,不用斜杠缩写让 Agent 猜测。
|
|
16
|
+
|
|
17
|
+
`tools: ""` 的 MCP-only 小探针不执行工作流入口,因此不要求安装元数据;不能用这类探针替代完整工作流对照。入口检查可通过 `node --test test/benchmark/claude-preflight.check.mjs` 免费验证。实跑中的失败、试错、资料重写和收尾请求全部保留,来源保真通过不等于收益成立。
|
|
18
|
+
|
|
19
|
+
`usage analyze` 与 `doctor --cost` 只读。`tools/experiments/cost-benchmark.mjs run|continue` 会启动真实模型会话,须由用户明确授权;不内置 token 或美元上限。原始提示词、源码和工具返回留在本地 JSONL,报告只给统计、问题和证据路径。
|
|
20
|
+
|
|
21
|
+
## Apply 的真实设计对照
|
|
22
|
+
|
|
23
|
+
`node tools/experiments/claude-apply-benchmark.mjs <config.json>` 为 apply 的独立实验入口,复用上述安装元数据、制品和输入摘要校验,按场景声明所需的外部适配器。它不属于正常 Worker 调度,也不修改用户全局模型或插件配置。
|
|
24
|
+
|
|
25
|
+
- 配置必须显式给出 `stage: "apply"`、`model`、`effort`、`claudeBinary`、`expectedCompletedTaskIds` 和 `expectedNode`(`code` 或 `gate_b`)。`measurement` 记录页面及其 task 集合,公共准备单列。
|
|
26
|
+
- 场景通过 scenario.requiredInputs 和 scenario.checks 声明冻结输入及启动前检查。脚本和断言归独立实验工程,不从宿主文件名推断工具链。成功摘要记入 request.pagePreflight;本地预检不代替编译与行为验收。
|
|
27
|
+
- 用户选择完成页面后比较时使用 `usagePolicy: "observe"`:必须省略 turns、美元、请求数和 token 限制,不传相应 CLI 上限;用量缺失或监控异常记证据,不因此终止页面。可省略 `timeoutMs`;页面按任务完成或真实阻塞结束。
|
|
28
|
+
- 只有获授权采用阈值停止时使用 `usagePolicy: "enforce"`(兼容旧配置的默认值):冻结正数 `maxTurns`、`maxBudgetUsd`、`timeoutMs`,以及正整数 `maxObservedModelRequests`、`maxObservedInputTokens`、`maxSingleRequestInputTokens`。美元参数是 CLI 估值限制,不代表已取得账单或证明活动子任务已限流。
|
|
29
|
+
- 保留生命周期 hooks;所需插件、MCP 服务和额外工具由 scenario.requiredPlugins、scenario.mcpServers 与 scenario.allowedTools 显式声明。
|
|
30
|
+
- 本入口开放实现所需的本地工具及场景声明的额外工具;通过会话级 `apply-worker`(`model: "inherit"`)执行节点,控制提示指定该类型且不传模型覆盖,避免误选全局代理路由。一个 code task 的 scout / build / verify / repair 各派发一次该子代理,每次只传对应节点文件与少量路径。固定工具列表不是文件系统沙箱,输入/改动范围仍需核查。
|
|
31
|
+
- 子进程设置 `CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0`,避免 controller 结束后 CLI 的后台等待上限中断未完成 worker,并把实际环境覆盖写入 request。控制会话优先前台等待 Agent;已有后台任务用 `TaskOutput` 等待,不能仅派发就结束。该环境开关须随固定 Claude 版本验证,不修改全局配置。
|
|
32
|
+
- 正式运行前验证实际 Worker 的 `get_design_context` 和 `get_screenshot`;注册成功或父会话 metadata 成功不能代替子 Worker 证据。保存原始失败,修复后在新输出目录重试。
|
|
33
|
+
- 两侧从同一宿主/plan/决定及前置代码快照开始,真实设计请求、截图和返工计入各侧成本;核对设计内容/版本一致性,变化的来源不能作为可比样本。
|
|
34
|
+
- runner 每秒读取本会话及原生子任务日志,按现有 message ID 去重规则合计请求与输入。仅 enforce 模式在达到用量限制、启动 60 秒无用量或监控失败时停止专属进程组并保存 `stopReason`;observe 模式持续记录。显式配置的运行超时、进程错误或本地输出缓冲超限另行报告。只能在用量回报后停止,可能超出一个或多个在途请求;不是账户美元硬上限。中断缺失的用量、缓存或最终费用保持 unknown,不自动续跑。
|
|
35
|
+
- `run.json` 的完成标志只核验 CLI 退出、被测 OpenSpec 插件、场景要求的插件出现、目标节点、任务完成状态及未触发停止;它不是业务或视觉质量结论。另行审查原生日志中的真实 Figma 调用、子 Agent 模型、截图和行为测试。共用前置准备单列,完整 apply 成本仍须包含它。
|
|
36
|
+
- 原生消息与 CLI `modelUsage` 的汇总分别保留,覆盖范围未核平时 `auxiliaryUsage` 为 null;不能把两者差额当辅助用量,更不能将负差额截成 0。缺失 CLI 汇总保持 null,`usageReconciliation` 明确记录未核平。
|
|
37
|
+
- 离线入口检查:`node --test test/benchmark/claude-apply-preflight.check.mjs test/benchmark/claude-apply-process.check.mjs test/benchmark/apply-session-metrics.check.mjs`。该检查不启动模型,也不证明 Figma 已登录或目标文件有权限。
|
|
38
|
+
- 运行结束后 `node tools/experiments/apply-session-metrics.mjs <run 目录>` 只读汇总会话级行为指标:控制会话 Read / 图片 Read / CLI 调用,各 Worker 阶段、首写前 Read、图片 Read、像素脚本、峰值单请求输入、≥200k 请求数、是否写出 result.json、与前序 Worker 的重复读取比例,并对照实验声明的目标给出逐项判定。图片与像素脚本按文件后缀和命令关键词识别,是启发式而非精确统计;`--json` 输出完整字段。
|
|
39
|
+
|
|
40
|
+
## 用量口径
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
openspec usage analyze --change example \
|
|
44
|
+
--session /evidence/controller-rollout.jsonl /evidence/child-rollout.jsonl \
|
|
45
|
+
--session /evidence/exec.jsonl \
|
|
46
|
+
--expect-model gpt-6-astra --expect-effort high --format md
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
- Codex `input_tokens` 已包含缓存输入,输出已包含 reasoning;缓存写入、推理明细缺失时记 unknown。不要把明细再叠加。
|
|
50
|
+
- 新版持久化日志有 `token_usage_record` 时按 `response_id` 去重,并与累计快照核对;旧版使用累计快照差分。快照重置分段,重复/冲突/尾部中断明确报告。
|
|
51
|
+
- 自动压缩按 `compacted.compaction_response_id` 关联真实请求,计入总量并单列 `compaction` 阶段。部分客户端的旧 `token_count` 快照不包含该请求;只有逐请求合计等于请求记录中的线程累计值,且旧快照与已识别压缩用量之和完全解释差额时才接受这种差异,并报告 `compaction_usage_reconciled`。相同响应 ID 的累计值冲突和未解释的缺口仍阻止比较;完全相同的重复记录只计一次。
|
|
52
|
+
- exec 的 `turn.completed.usage` 按每轮增量处理,不能当线程累计快照做差;这一语义与 [Codex 原生事件定义](https://github.com/openai/codex/blob/main/codex-rs/exec/src/exec_events.rs) 一致。多个恢复执行导出缺少稳定 turn ID 时必须补 rollout,不能据此宣称总量完整。
|
|
53
|
+
- 同线程 rollout 优先于 exec 导出,重复文件/重复事件不重复收费。冲突时保留已知差额为未归属用量,并禁止比较。调用次数可由请求记录证明时才给数字;工具调用数、用量观测数独立统计。
|
|
54
|
+
- 必须提供控制会话、全部实际子线程及独立执行、失败与重试日志。缺子线程、缺终止记录、计数歧义会使 `tokenComplete=false`。旧 exec JSON 缺模型/effort 时补持久化 rollout。
|
|
55
|
+
- `--events <events.jsonl>` 可补充阶段归属。无法归属的消耗仍计入总量。启发式阶段归属不替代线程总量。
|
|
56
|
+
- Claude 使用独立公式:input + cache read + cache creation。流式消息跨文件按 message ID 合并;完整输入/缓存分项覆盖临时输入值,不能逐字段取最大值后将临时总输入与最终缓存相加。完整分项尚未出现时保留部分字段并标记覆盖缺口;输出保留累计最大值。继承消息归属已知父级发起会话,tool-use ID 只计一次。反序导入不改变结果。父级关联不足时保留用量并报告归属缺口。synthetic 本地消息不算模型请求;synthetic API error 的零值不证明失败调用免费,记录为计量缺口。
|
|
57
|
+
- `comparisonEligible` 只说明这批日志的用量/模型信息满足比较前提;不代表来源一致、隔离已证明、质量通过或降本达标。报告还须核对实验清单与质量证据。
|
|
58
|
+
|
|
59
|
+
费用默认 `unavailable`。ChatGPT 登录不是逐次美元账单。传入有来源的实际账单 `--billing` 或指定费率 `--rates` 才计算;二者不要混为同一口径:
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{"source":"账单或费率文件的可核查位置","models":{"model-name":{"inputPerMillion":0,"cachedInputPerMillion":0,"cacheWriteInputPerMillion":0,"outputPerMillion":0}}}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
以上零值仅展示字段,必须换成有依据的费率。账单格式为 `{"source":"...","charges":[{"threadId":"...","usd":1.23}]}`。模型/缓存/收费项缺失时 `complete=false`,不能宣布美元达标。
|
|
66
|
+
|
|
67
|
+
## 场景配置与能力诊断
|
|
68
|
+
|
|
69
|
+
专用工具、资源格式、模型、服务地址、检查脚本和质量断言由独立实验工程保存。新场景显式声明:
|
|
70
|
+
|
|
71
|
+
```json
|
|
72
|
+
{
|
|
73
|
+
"scenario": {
|
|
74
|
+
"requiredInputs": ["experiment/inputs.json", "scripts/check-inputs.mjs"],
|
|
75
|
+
"checks": [{"command": "node", "args": ["scripts/check-inputs.mjs", "experiment/inputs.json"]}],
|
|
76
|
+
"requiredPlugins": [],
|
|
77
|
+
"allowedTools": [],
|
|
78
|
+
"mcpServers": {},
|
|
79
|
+
"forbiddenPromptPatterns": []
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
requiredInputs 必须包含在冻结 inputs 摘要中;checks 在启动模型前以 argv 执行,失败或超时终止启动。requiredPlugins 要求会话设置及运行回执都出现所选插件;mcpServers 将场景内命名服务绑定到明确 URL。提示歧义规则由场景给出 pattern/message,运行器不识别特定生成器命令。旧 pageInputManifest 配置需将原 manifest 和真实验证命令迁入场景,不能静默丢弃原检查。
|
|
85
|
+
|
|
86
|
+
`doctor --cost` 从项目 plugins、解析成功的 use 引用及显式实验要求生成能力集合。额外能力通过重复 `--capability <name>` 或证据文件 requiredCapabilities 数组声明;没有选择的插件不报缺失。用量、探针文件摘要与隔离证据不足仍保持 gap。
|
|
87
|
+
|
|
88
|
+
国际化翻译、增量合入与校验使用宿主选择的插件入口,检查历史值、完整语言覆盖、占位符和失败日志。插件实现、实验结论和真实宿主服务地址不由框架文档重复维护。
|