@cat-factory/app 0.213.1 → 0.215.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +107 -13
- package/app/components/observability/StepMetricsBar.vue +11 -0
- package/app/components/panels/AgentStepDetail.vue +14 -0
- package/app/components/panels/MergerResultView.vue +20 -2
- package/app/components/panels/ObservabilityPanel.vue +57 -0
- package/app/components/panels/ResultWindowShell.vue +77 -0
- package/app/components/panels/StepReproductionReport.vue +167 -0
- package/app/components/pipeline/BinaryOutputStepPicker.vue +25 -0
- package/app/components/tutorial/TutorialCatalogue.vue +14 -1
- package/app/components/tutorial/TutorialNudge.vue +107 -0
- package/app/components/tutorial/TutorialOverlay.vue +92 -11
- package/app/composables/api/execution.ts +5 -2
- package/app/composables/api/tutorial.ts +25 -0
- package/app/composables/useApi.ts +2 -0
- package/app/composables/usePipelineErrorToast.ts +4 -0
- package/app/composables/useTutorialNudge.ts +77 -0
- package/app/composables/useTutorialSync.ts +141 -0
- package/app/modular/external-tools.spec.ts +1 -0
- package/app/modular/nav-contributions.spec.ts +2 -0
- package/app/modular/nav-contributions.ts +11 -0
- package/app/modular/nav-gates.ts +10 -0
- package/app/modular/registry.spec.ts +1 -0
- package/app/modular/tutorial-tours.spec.ts +55 -4
- package/app/modular/tutorial-tours.ts +231 -9
- package/app/pages/index.vue +20 -1
- package/app/stores/tutorial.prompt.ts +59 -0
- package/app/stores/tutorial.record.ts +191 -0
- package/app/stores/tutorial.spec.ts +207 -0
- package/app/stores/tutorial.ts +78 -91
- package/app/stores/workspace/hydrate.ts +5 -0
- package/app/types/domain.ts +3 -0
- package/app/types/reproduction.ts +11 -0
- package/app/utils/binaryOutput.spec.ts +56 -2
- package/app/utils/binaryOutput.ts +55 -32
- package/app/utils/observability.spec.ts +44 -1
- package/app/utils/observability.ts +50 -0
- package/app/utils/reproduction.ts +51 -0
- package/app/utils/tutorial.spec.ts +255 -0
- package/app/utils/tutorial.ts +173 -0
- package/i18n/locales/de.json +148 -6
- package/i18n/locales/en.json +153 -6
- package/i18n/locales/es.json +148 -6
- package/i18n/locales/fr.json +148 -6
- package/i18n/locales/he.json +148 -6
- package/i18n/locales/it.json +148 -6
- package/i18n/locales/ja.json +148 -6
- package/i18n/locales/pl.json +148 -6
- package/i18n/locales/tr.json +148 -6
- package/i18n/locales/uk.json +148 -6
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -208,6 +208,54 @@ omission. It thins an offer, never the library: an un-offered tour is listed, st
|
|
|
208
208
|
in the progress line and one footer button away, and `requires` remains the only thing that can
|
|
209
209
|
hold a tour back, which is always reported.
|
|
210
210
|
|
|
211
|
+
**The finish card HANDS OFF to the next walkthrough** (`nextTourAfter`, offered beside Done). The
|
|
212
|
+
delivery loop is a chain — each tour produces the state the next one requires — and finishing one is
|
|
213
|
+
the last moment the product can bring the tutorial up at all: `startTour` writes
|
|
214
|
+
`decision: 'accepted'`, which is exactly what stops the launch prompt auto-opening, so without this
|
|
215
|
+
the walkthrough a user's own action just unlocked is reachable only by going and finding the
|
|
216
|
+
catalogue. It offers ONE tour, launch-offer tours first whatever their `order` (a deployment's
|
|
217
|
+
reference tour must not cut into the arc), never one already completed or the one just finished, and
|
|
218
|
+
nothing at all when nothing is ready — where the plain Done is the honest ending. It reads the gates
|
|
219
|
+
LIVE, which is the one deliberate exception to the held-script rule below: completing `first-task` is
|
|
220
|
+
precisely what makes `run-task` ready, so a candidate resolved at tour start would be empty exactly
|
|
221
|
+
when it matters. Taking the offer completes this tour first (or its badge stays "not started") and
|
|
222
|
+
goes through the same `useTutorialLaunch().launch`, so a suggested tour the user had broken off
|
|
223
|
+
earlier RESUMES. **Per-run overlay state resets with the script**, not by the component unmounting:
|
|
224
|
+
the handoff completes one tour and starts the next in ONE tick, so `touring` never goes false for a
|
|
225
|
+
render and the finished tour's skips would otherwise be counted against the new one.
|
|
226
|
+
|
|
227
|
+
**A CONTEXTUAL offer catches a tour becoming takeable** (`resolveNudge` over `newlyAvailableTour` +
|
|
228
|
+
`useTutorialNudge` + `TutorialNudge.vue`): a corner card, not a modal, since the whole point is the
|
|
229
|
+
moment. The trigger is deliberately not a per-surface hook — every tour already declares, as its
|
|
230
|
+
`requires`, the predicate that means "you can take this now", so ONE rule over the resolved
|
|
231
|
+
catalogue covers the catalog and inherits `navRequirementDrift` unchanged. Four rules bind it. It
|
|
232
|
+
fires on a TRANSITION into `ready`, never on the standing state, which would greet every board load
|
|
233
|
+
with an offer about a walkthrough available for weeks. Only the launch-offer arc, for the same reason
|
|
234
|
+
`offeredAtLaunch` exists. Never twice per tour (`nudgedTourIds`, persisted) and never after an
|
|
235
|
+
explicit decline — "no thanks" answered the question about guided tours, not about when it was
|
|
236
|
+
asked. And the offer is HELD rather than dropped while a tour or a tutorial window is up, because
|
|
237
|
+
the two most valuable moments (a run parked, a run failed) routinely arrive then; it is marked spent
|
|
238
|
+
when RAISED, so holding it cannot become nagging.
|
|
239
|
+
|
|
240
|
+
**What that transition is measured against is the subtle half, and it takes TWO guards** (both in
|
|
241
|
+
the pure `resolveNudge`, so they are unit-tested rather than inferred from a watcher; the composable
|
|
242
|
+
holds only the ref, because a pure function cannot). Every gate reads a store something fills
|
|
243
|
+
asynchronously, so a baseline taken when the composable mounts records "nothing is takeable" and the
|
|
244
|
+
app's own startup then reads as a transition, which is the every-board-load greeting the rule exists
|
|
245
|
+
to prevent arriving through the mechanism meant to be its cure.
|
|
246
|
+
|
|
247
|
+
- **`workspace.ready`** gates taking a baseline at all on the snapshot having been fanned out, and
|
|
248
|
+
is re-set per board, which is what makes switching boards RE-SEED rather than offer everything the
|
|
249
|
+
incoming board happens to satisfy.
|
|
250
|
+
- **A board-state FINGERPRINT** (`boardStateFingerprint`, over the `boardHas*` gates) is what an
|
|
251
|
+
offer requires to have MOVED. Readiness widening is not the world changing: a permission
|
|
252
|
+
resolving or a capability probe answering makes tours takeable that were "blocked" only because
|
|
253
|
+
the app had not found out yet, and the app finding out about itself is not a moment to interrupt
|
|
254
|
+
anyone about. Those resolutions advance the baseline silently. This is the guard that generalises
|
|
255
|
+
— it needs no list of which stores load late, because none of them describe the world — and it is
|
|
256
|
+
the reason readiness alone was not enough: `workspace.ready` flips before the RBAC access and the
|
|
257
|
+
integration probes have landed.
|
|
258
|
+
|
|
211
259
|
**The catalogue lists the tours it CANNOT start, and says what would unlock each.** That is the
|
|
212
260
|
reason a tour's preconditions are declared (`TutorialRequirement`: an id, a copy key, and the
|
|
213
261
|
gate predicate) rather than being an anonymous `when(gates)`. A predicate can only answer "no",
|
|
@@ -226,7 +274,42 @@ is what the prompt and the overlay have always seen) and `catalogue` (everything
|
|
|
226
274
|
reads the SAME registered `gates` service the nav filter does, through the shared-dependency
|
|
227
275
|
`useOptional('gates')`, so the two can never disagree about what this board offers.
|
|
228
276
|
|
|
229
|
-
Progress
|
|
277
|
+
**Progress follows the USER, not the browser** (`useTutorialSync` / `useTutorialServer` over
|
|
278
|
+
`GET|PUT|DELETE /tutorial/progress`). The browser-persisted store stays what the SPA reads and stays
|
|
279
|
+
fully functional with no accounts, no store wired on the facade, or offline; the server row is a
|
|
280
|
+
MIRROR, adopted in the snapshot fan-out (`stores/workspace/hydrate.ts`) so the launch prompt decides
|
|
281
|
+
whether to appear against the merged state rather than this browser's copy alone. Both id lists are
|
|
282
|
+
grow-only sets and are UNIONED on BOTH sides, because two browsers signed in as one person each hold
|
|
283
|
+
a full copy and each write it back: a last-writer-wins replace on either side silently drops what the
|
|
284
|
+
other learned, and the symptom is a finished walkthrough going back to "not started" days later.
|
|
285
|
+
Only `decision` is replaced (a preference, not an accumulating fact), and then only where this browser
|
|
286
|
+
is not holding an answer the mirror has not carried yet: without that exception a failed push lets the
|
|
287
|
+
next snapshot re-adopt the older server answer, so "No thanks" silently comes back as accepted and
|
|
288
|
+
every contextual offer re-arms. "Reset progress" is a DELETE, which is also why the catalogue calls
|
|
289
|
+
`useTutorialServer` and not just the store — a local clear alone would be undone by the next snapshot
|
|
290
|
+
re-merging the row.
|
|
291
|
+
|
|
292
|
+
Three rules make fire-and-forget honest rather than merely convenient. Every push carries the WHOLE
|
|
293
|
+
local state, so a retry, a racing tab and a stale copy are all the same well-formed write. The
|
|
294
|
+
RESPONSE (the merged row) is reconciled back through the store, which is what closes the hole the
|
|
295
|
+
server's un-rev-guarded merge leaves: two concurrent merges can lose a writer's ids, because a union
|
|
296
|
+
is idempotent under retry but not commutative under concurrency, and the loser's answer comes back
|
|
297
|
+
missing something local and re-pushes automatically. And the mirror watches the store's LOCAL
|
|
298
|
+
revision counter rather than its state, because adopting the server's own ids is a state change too:
|
|
299
|
+
watching the state posts the server's row straight back at it on every fresh-browser board load, and
|
|
300
|
+
a reset (whose server side is the DELETE) would race a push of the freshly-emptied state.
|
|
301
|
+
|
|
302
|
+
**The funnel is counted** (`POST /tutorial/events` → the kernel `OperationalMetrics` counters
|
|
303
|
+
`tutorial.tour_started` / `_completed` / `_abandoned`, dimensioned by tour). The events are DERIVED
|
|
304
|
+
from the cursor in one watcher rather than emitted from each store action, because those are five
|
|
305
|
+
sites and a missing `started` fails nothing — it just biases the number the next decision is made
|
|
306
|
+
against. Vue's batching is what makes that work across the handoff: it completes one tour and starts
|
|
307
|
+
the next in a single tick, so the cursor goes `A → null → B` and the watcher sees `A → B` with the
|
|
308
|
+
completion list one longer, reporting "A completed, B started". A resume counts as a start, which is
|
|
309
|
+
deliberate: an attempt is an attempt, and not counting re-entries would make completions exceed
|
|
310
|
+
starts. Nothing per-user or per-workspace is recorded, and nothing is stored.
|
|
311
|
+
|
|
312
|
+
Progress is per tour id. The catalogue's counter is over the WHOLE catalog, not
|
|
230
313
|
the runnable part: counting only today's runnable tours would move the denominator every time a
|
|
231
314
|
repo was linked, and "2 of 2 completed" on a board with four walkthroughs still waiting reads as
|
|
232
315
|
a finished tutorial. `Reset progress` clears the completions, the resume point AND the saved
|
|
@@ -245,10 +328,9 @@ state by design: it is openable mid-tour, which is what the `continue` action is
|
|
|
245
328
|
is SUPPRESSED rather than unmounted, because it holds the running tour's resolved script and a
|
|
246
329
|
remount would re-resolve it against gates that may have flipped since the tour started.
|
|
247
330
|
|
|
248
|
-
The
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
section is the authority on how the thing WORKS.
|
|
331
|
+
The decisions behind this surface, and why each alternative was rejected, are recorded in
|
|
332
|
+
[ADR 0033](../../backend/docs/adr/0033-in-app-tutorials.md). This section is the authority on how
|
|
333
|
+
the thing WORKS.
|
|
252
334
|
|
|
253
335
|
A tour is **data, not components**: an ordered list of steps, each pointing at an on-screen
|
|
254
336
|
control by its `data-testid` (the e2e anchor vocabulary; cover a control that has none by
|
|
@@ -302,14 +384,18 @@ each drift on their own: the same split components make for inline placeholders.
|
|
|
302
384
|
The built-ins come in two halves. The DELIVERY LOOP, end to end, each tour gated on the state the
|
|
303
385
|
previous one leaves behind, so the launch prompt only ever offers what this board can demonstrate:
|
|
304
386
|
board basics, add a repository (`add-service`), create a task (`first-task`), run it (`run-task`),
|
|
305
|
-
answer it when it parks (`answer-park`),
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
387
|
+
answer it when it parks (`answer-park`), read a failure when one comes (`diagnose-failure`), review
|
|
388
|
+
and merge the result (`review-merge`). The loop covers work going WRONG on purpose: a first run fails
|
|
389
|
+
often, `boardHasFinishedRun` deliberately excludes failures (a result view and a merge control are
|
|
390
|
+
not what a failed run renders), and that left the state a new user is most likely to be in as the
|
|
391
|
+
only one on the arc with no walkthrough. Then the PLATFORM behind it, catalogue-only: connect an
|
|
392
|
+
engine (`wire-models`), assemble a flow (`design-pipeline`), curate the standards agents read
|
|
393
|
+
(`agent-standards`), link the systems a run talks to (`connect-systems`), where runs execute
|
|
394
|
+
(`prepare-infrastructure`), review by panel (`panel-reviews`), the shared services designs build on
|
|
395
|
+
(`share-services`). Each of those covers ONE surface and ends there, because the surface opens as a
|
|
396
|
+
modal over the sidebar it was reached from, so a later step could not click another sidebar entry
|
|
397
|
+
anyway, and each declares exactly the permission that renders the entry it clicks, since a weaker
|
|
398
|
+
requirement offers a tour to someone with no such control and it then reports itself abridged.
|
|
313
399
|
|
|
314
400
|
**That pairing is DERIVED from the nav catalog, not restated.** A step whose anchor IS a nav
|
|
315
401
|
entry's `testId` is checked by `navRequirementDrift` (`tutorial-tours.spec.ts`, beside the anchor
|
|
@@ -322,6 +408,14 @@ from BASIC mode; basic is the shipped default, so that tour would be offered to
|
|
|
322
408
|
and find nothing. "Renders" therefore means the gate AND the tier, and a tour that wants an
|
|
323
409
|
advanced entry has to declare the tier as a requirement of its own.
|
|
324
410
|
|
|
411
|
+
**A SECTION can hide itself the same way, one level below anything that guard can see.**
|
|
412
|
+
`panel-reviews` clicks `nav-model-config`, a BASIC-mode entry, but the consensus section inside that
|
|
413
|
+
panel renders on `uiMode.isAdvanced || groups.hasGroups` — so on the shipped default tier a workspace
|
|
414
|
+
that has never made a group renders nothing for the anchored step to find. `navRequirementDrift` pairs
|
|
415
|
+
a tour only against a NAV entry's visibility, so this one is declared by hand (`advancedTier`) and
|
|
416
|
+
pinned by its own named case in `tutorial-tours.spec.ts`. Any tour anchoring INSIDE a surface owes the
|
|
417
|
+
same check of what that surface's own `v-if`s read.
|
|
418
|
+
|
|
325
419
|
Two deliberate asymmetries about click-to-advance: `run-task` points at Start without it,
|
|
326
420
|
because starting a run spends real model budget and nobody should discover they agreed to that by
|
|
327
421
|
following a tutorial; `design-pipeline` points at Save without it, because Save is DISABLED until
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
import { computed } from 'vue'
|
|
3
3
|
import type { StepMetrics } from '~/types/execution'
|
|
4
4
|
import {
|
|
5
|
+
formatCost,
|
|
5
6
|
formatMs,
|
|
6
7
|
formatTokens,
|
|
7
8
|
headroomColor,
|
|
@@ -33,6 +34,10 @@ const totalInput = computed(() => totalInputTokens(m.value))
|
|
|
33
34
|
const cacheRead = computed(() => m.value.cacheReadTokens ?? 0)
|
|
34
35
|
const cacheWrite = computed(() => m.value.cacheWriteTokens ?? 0)
|
|
35
36
|
const hasCache = computed(() => cacheRead.value > 0 || cacheWrite.value > 0)
|
|
37
|
+
// Money beside the volume, not instead of it. Null when the deployment prices nothing or has
|
|
38
|
+
// no rate for the model that ran — the figure is then OMITTED rather than shown as 0.00, which
|
|
39
|
+
// would claim the step was free.
|
|
40
|
+
const cost = computed(() => formatCost(m.value.costEstimate, m.value.costCurrency))
|
|
36
41
|
const headroom = computed(() => headroomRatio(m.value))
|
|
37
42
|
const transport = computed(() => transportRatio(m.value))
|
|
38
43
|
const headroomTone = computed(() => headroomColor(headroom.value, m.value.truncatedCalls > 0))
|
|
@@ -61,6 +66,12 @@ const headroomTone = computed(() => headroomColor(headroom.value, m.value.trunca
|
|
|
61
66
|
>
|
|
62
67
|
{{ formatTokens(totalInput) }}↑ {{ formatTokens(m.completionTokens) }}↓
|
|
63
68
|
</span>
|
|
69
|
+
<template v-if="cost">
|
|
70
|
+
<span class="text-slate-500">·</span>
|
|
71
|
+
<span class="tabular-nums text-slate-300" :title="t('observability.metricsBar.costHint')">
|
|
72
|
+
{{ cost }}
|
|
73
|
+
</span>
|
|
74
|
+
</template>
|
|
64
75
|
<div class="ms-auto flex items-center gap-1">
|
|
65
76
|
<UBadge v-if="m.errors > 0" color="error" variant="subtle" size="sm">
|
|
66
77
|
{{ t('observability.metricsBar.errors', { count: m.errors }, m.errors) }}
|
|
@@ -7,6 +7,7 @@ import StepRestartControl from '~/components/panels/StepRestartControl.vue'
|
|
|
7
7
|
import StepMetadataCard from '~/components/panels/StepMetadataCard.vue'
|
|
8
8
|
import StepTestReport from '~/components/panels/StepTestReport.vue'
|
|
9
9
|
import StepEffortReport from '~/components/panels/StepEffortReport.vue'
|
|
10
|
+
import StepReproductionReport from '~/components/panels/StepReproductionReport.vue'
|
|
10
11
|
import StepFragmentAdherence from '~/components/panels/StepFragmentAdherence.vue'
|
|
11
12
|
import BinaryOutputReport from '~/components/binaryOutput/BinaryOutputReport.vue'
|
|
12
13
|
import EnvironmentStatusPanel from '~/components/environments/EnvironmentStatusPanel.vue'
|
|
@@ -558,6 +559,19 @@ async function copyOutput() {
|
|
|
558
559
|
effectiveness, key obstacles). Only when the agent reported one. -->
|
|
559
560
|
<StepEffortReport v-if="step.effortReport" :report="step.effortReport" />
|
|
560
561
|
|
|
562
|
+
<!-- the bugfix REPRODUCTION PROOF: the declared reproducing check run against the
|
|
563
|
+
pre-fix tree and the final one, with both captured outputs, or the agent's
|
|
564
|
+
structural declaration that the bug cannot be reproduced. This panel is the
|
|
565
|
+
half that matters most: the engine records the proof on whichever step OPENED
|
|
566
|
+
the pull request, and in every built-in pipeline that is the `coder` — a kind
|
|
567
|
+
with no dedicated result view, so it opens HERE and the result-window shell is
|
|
568
|
+
never involved. Self-hiding for a run that declared no reproducing check. -->
|
|
569
|
+
<StepReproductionReport
|
|
570
|
+
v-if="step.reproduction"
|
|
571
|
+
:report="step.reproduction"
|
|
572
|
+
variant="card"
|
|
573
|
+
/>
|
|
574
|
+
|
|
561
575
|
<!-- what the step declared it stored through a foundational storage service, and
|
|
562
576
|
every way that record is incomplete. This panel is the OTHER half of the
|
|
563
577
|
result-window shell's trailing section: a step whose kind declares no
|
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
// banner — instead of the agent's raw JSON. Opened via the universal result-view host,
|
|
8
8
|
// the same seam the requirements / tester windows use.
|
|
9
9
|
import { computed } from 'vue'
|
|
10
|
-
import type { ChangeClass, MergeAxis, MergeDecision } from '@cat-factory/contracts'
|
|
10
|
+
import type { ChangeClass, MergeAxis, MergeDecision, WorkspaceRole } from '@cat-factory/contracts'
|
|
11
11
|
import StepRunMeta from '~/components/panels/StepRunMeta.vue'
|
|
12
12
|
import ResultWindowShell from '~/components/panels/ResultWindowShell.vue'
|
|
13
13
|
import MarkdownProse from '~/components/common/MarkdownProse.vue'
|
|
@@ -70,6 +70,8 @@ const REASON_KEYS: Record<MergeDecision['reason'], string> = {
|
|
|
70
70
|
merge_partial: 'panels.mergerResult.reason.merge_partial',
|
|
71
71
|
class_auto_merge: 'panels.mergerResult.reason.class_auto_merge',
|
|
72
72
|
class_requires_review: 'panels.mergerResult.reason.class_requires_review',
|
|
73
|
+
role_requires_review: 'panels.mergerResult.reason.role_requires_review',
|
|
74
|
+
dry_run: 'panels.mergerResult.reason.dry_run',
|
|
73
75
|
}
|
|
74
76
|
const OUTCOME_KEYS: Record<MergeDecision['outcome'], string> = {
|
|
75
77
|
auto_merged: 'panels.mergerResult.outcome.auto_merged',
|
|
@@ -131,7 +133,18 @@ const axes = computed(() => {
|
|
|
131
133
|
]
|
|
132
134
|
})
|
|
133
135
|
|
|
134
|
-
/**
|
|
136
|
+
/**
|
|
137
|
+
* The role the run was admitted under, when one was pinned. A `role_requires_review` line names
|
|
138
|
+
* it, because the remedy that reason points at is a PERSON (a teammate on a higher tier can merge
|
|
139
|
+
* this as it stands) rather than a setting.
|
|
140
|
+
*/
|
|
141
|
+
const ROLE_KEYS: Record<WorkspaceRole, string> = {
|
|
142
|
+
admin: 'merge.role.admin',
|
|
143
|
+
member: 'merge.role.member',
|
|
144
|
+
viewer: 'merge.role.viewer',
|
|
145
|
+
}
|
|
146
|
+
|
|
147
|
+
/** The plain-language "why" line, interpolating the preset, any exceeded axes, and the role. */
|
|
135
148
|
const reasonText = computed(() => {
|
|
136
149
|
const d = decision.value
|
|
137
150
|
if (!d) return ''
|
|
@@ -139,6 +152,11 @@ const reasonText = computed(() => {
|
|
|
139
152
|
return t(REASON_KEYS[d.reason], {
|
|
140
153
|
preset: d.thresholds.presetName,
|
|
141
154
|
axes: axisLabels,
|
|
155
|
+
// Never blank: a role-scoped reason is only ever produced for a run that pinned one, and the
|
|
156
|
+
// fallback keeps the sentence readable rather than leaving a hole if that ever changes.
|
|
157
|
+
role: d.thresholds.initiatorRole
|
|
158
|
+
? t(ROLE_KEYS[d.thresholds.initiatorRole])
|
|
159
|
+
: t('merge.role.unattributed'),
|
|
142
160
|
})
|
|
143
161
|
})
|
|
144
162
|
</script>
|
|
@@ -10,9 +10,11 @@ import type {
|
|
|
10
10
|
import { agentKindMeta } from '~/utils/catalog'
|
|
11
11
|
import {
|
|
12
12
|
foldRunPhaseMetrics,
|
|
13
|
+
formatCost,
|
|
13
14
|
formatMs,
|
|
14
15
|
formatTokens,
|
|
15
16
|
pct,
|
|
17
|
+
sumCosts,
|
|
16
18
|
totalInputTokens,
|
|
17
19
|
} from '~/utils/observability'
|
|
18
20
|
|
|
@@ -164,6 +166,36 @@ const phaseRows = computed(() => foldRunPhaseMetrics(instance.value?.steps ?? []
|
|
|
164
166
|
const phaseCarryTotal = computed(() =>
|
|
165
167
|
phaseRows.value.reduce((acc, p) => acc + p.carryCostTokens, 0),
|
|
166
168
|
)
|
|
169
|
+
/**
|
|
170
|
+
* The currency the engine priced this run in. Read off the step rollups rather than assumed,
|
|
171
|
+
* because the amounts come from a deployment-configured table whose currency an operator sets;
|
|
172
|
+
* absent ⇒ nothing priced the run, and every amount below is null too.
|
|
173
|
+
*/
|
|
174
|
+
const costCurrency = computed(
|
|
175
|
+
() => instance.value?.steps?.find((s) => s.metrics?.costCurrency)?.metrics?.costCurrency,
|
|
176
|
+
)
|
|
177
|
+
/**
|
|
178
|
+
* Whether to show money at all: this deployment prices, and at least one phase of this run
|
|
179
|
+
* actually got a figure.
|
|
180
|
+
*
|
|
181
|
+
* Deliberately NOT gated on the run TOTAL being known. A mixed-model run is the normal shape
|
|
182
|
+
* (a harness CLI serves some turns with a model of its own choosing), so one phase on an
|
|
183
|
+
* unpriced model is common — and gating the column on the total meant that one phase hid the
|
|
184
|
+
* cost of every other, with no indication anything had been withheld.
|
|
185
|
+
*/
|
|
186
|
+
const showCost = computed(
|
|
187
|
+
() => !!costCurrency.value && phaseRows.value.some((p) => p.costEstimate != null),
|
|
188
|
+
)
|
|
189
|
+
/**
|
|
190
|
+
* The run's estimated cost, folded from the same SQL rollup the phase table shows — NOT from
|
|
191
|
+
* the capped call list the token totals beside it use, which would silently under-report a run
|
|
192
|
+
* longer than the page. Null when any phase could not be priced (see `sumCosts`), in which case
|
|
193
|
+
* the tile SAYS the total is incomplete rather than quietly dropping it: a missing figure and a
|
|
194
|
+
* partial one are both wrong to render as a number, but only one of them is worth explaining.
|
|
195
|
+
*/
|
|
196
|
+
const runCost = computed(() =>
|
|
197
|
+
formatCost(sumCosts(phaseRows.value.map((p) => p.costEstimate)), costCurrency.value),
|
|
198
|
+
)
|
|
167
199
|
/** Share of the run's carry cost a phase accounts for (0..100), or null when nothing carried. */
|
|
168
200
|
function carryShare(carryCostTokens: number): number | null {
|
|
169
201
|
return phaseCarryTotal.value > 0 ? pct(carryCostTokens / phaseCarryTotal.value) : null
|
|
@@ -314,6 +346,21 @@ function exportJson() {
|
|
|
314
346
|
</dt>
|
|
315
347
|
<dd class="mt-0.5 tabular-nums text-slate-200">{{ totals.calls }}</dd>
|
|
316
348
|
</div>
|
|
349
|
+
<div v-if="showCost">
|
|
350
|
+
<dt class="text-[11px] uppercase tracking-wide text-slate-500">
|
|
351
|
+
{{ t('observability.summary.cost') }}
|
|
352
|
+
</dt>
|
|
353
|
+
<dd class="mt-0.5 tabular-nums text-slate-200">
|
|
354
|
+
{{ runCost ?? '—' }}
|
|
355
|
+
<span class="mt-0.5 block text-[11px] text-slate-500">
|
|
356
|
+
{{
|
|
357
|
+
runCost
|
|
358
|
+
? t('observability.summary.costHint')
|
|
359
|
+
: t('observability.summary.costIncomplete')
|
|
360
|
+
}}
|
|
361
|
+
</span>
|
|
362
|
+
</dd>
|
|
363
|
+
</div>
|
|
317
364
|
<div>
|
|
318
365
|
<dt class="text-[11px] uppercase tracking-wide text-slate-500">
|
|
319
366
|
{{ t('observability.summary.tokensInOut') }}
|
|
@@ -437,6 +484,11 @@ function exportJson() {
|
|
|
437
484
|
<th class="py-1 px-3 text-end font-normal">
|
|
438
485
|
{{ t('observability.phase.columns.tokensInOut') }}
|
|
439
486
|
</th>
|
|
487
|
+
<th v-if="showCost" class="py-1 px-3 text-end font-normal">
|
|
488
|
+
<span :title="t('observability.phase.costHint')">
|
|
489
|
+
{{ t('observability.phase.columns.cost') }}
|
|
490
|
+
</span>
|
|
491
|
+
</th>
|
|
440
492
|
<!-- The sort key, MARKED as one. Rows lead with carry cost rather than
|
|
441
493
|
with tokens, and the two orders genuinely differ: a phase that runs
|
|
442
494
|
late carries almost nothing however much it spent (nothing after it
|
|
@@ -476,6 +528,11 @@ function exportJson() {
|
|
|
476
528
|
{{ formatTokens(totalInputTokens(p)) }}↑
|
|
477
529
|
{{ formatTokens(p.completionTokens) }}↓
|
|
478
530
|
</td>
|
|
531
|
+
<td v-if="showCost" class="py-1.5 px-3 text-end tabular-nums text-slate-300">
|
|
532
|
+
<!-- An em dash, not 0: this phase's model had no rate, and a zero here
|
|
533
|
+
would read as a phase that cost nothing. -->
|
|
534
|
+
{{ formatCost(p.costEstimate, costCurrency) ?? '—' }}
|
|
535
|
+
</td>
|
|
479
536
|
<td class="py-1.5 ps-3 text-end tabular-nums text-slate-300">
|
|
480
537
|
{{ formatTokens(p.carryCostTokens) }}
|
|
481
538
|
<span v-if="carryShare(p.carryCostTokens) !== null" class="text-slate-600">
|
|
@@ -25,7 +25,9 @@ import { useModalBehavior } from '@modular-vue/core'
|
|
|
25
25
|
import StepRestartControl from '~/components/panels/StepRestartControl.vue'
|
|
26
26
|
import StepEffortReport from '~/components/panels/StepEffortReport.vue'
|
|
27
27
|
import StepValidationReport from '~/components/panels/StepValidationReport.vue'
|
|
28
|
+
import StepReproductionReport from '~/components/panels/StepReproductionReport.vue'
|
|
28
29
|
import BinaryOutputReport from '~/components/binaryOutput/BinaryOutputReport.vue'
|
|
30
|
+
import { REPRODUCTION_STATUS_KEYS } from '~/utils/reproduction'
|
|
29
31
|
import {
|
|
30
32
|
BINARY_OUTPUT_STATE_KEYS,
|
|
31
33
|
binaryOutputHasWarnings,
|
|
@@ -146,6 +148,34 @@ watch(
|
|
|
146
148
|
},
|
|
147
149
|
{ immediate: true },
|
|
148
150
|
)
|
|
151
|
+
/**
|
|
152
|
+
* The step's BUGFIX REPRODUCTION PROOF — resolved off the active step for the same reason as the
|
|
153
|
+
* three sections around it: the engine writes it onto whichever step OPENED the pull request, which
|
|
154
|
+
* is a property of the dispatch rather than of the step's own kind, so no window may be able to opt
|
|
155
|
+
* out of showing it. Absent for every run that declared no reproducing check, and the section
|
|
156
|
+
* disappears.
|
|
157
|
+
*/
|
|
158
|
+
const reproductionReport = computed(() => activeStep.value?.reproduction ?? null)
|
|
159
|
+
const reproductionOpen = ref(false)
|
|
160
|
+
// Anything short of proof opens expanded: an `inconclusive` verdict and a structural infeasibility
|
|
161
|
+
// declaration are both things a reviewer has to read and weigh, where `reproduced` is the one
|
|
162
|
+
// answer a collapsed one-line row states completely.
|
|
163
|
+
watch(
|
|
164
|
+
reproductionReport,
|
|
165
|
+
(report) => {
|
|
166
|
+
if (report && report.status !== 'reproduced') reproductionOpen.value = true
|
|
167
|
+
},
|
|
168
|
+
{ immediate: true },
|
|
169
|
+
)
|
|
170
|
+
/**
|
|
171
|
+
* The verdict's copy, icon and tone, from the EXHAUSTIVE lookup keyed off the contracts union
|
|
172
|
+
* rather than a t() call over a key assembled at runtime: the typed-key check cannot see such a
|
|
173
|
+
* key, so a fourth verdict would ship as a blank chip on the surface whose whole job is saying what
|
|
174
|
+
* was and was not proven.
|
|
175
|
+
*/
|
|
176
|
+
const reproductionKeys = computed(() =>
|
|
177
|
+
reproductionReport.value ? REPRODUCTION_STATUS_KEYS[reproductionReport.value.status] : null,
|
|
178
|
+
)
|
|
149
179
|
/**
|
|
150
180
|
* The step's BINARY-OUTPUT record — the third universal trailing section, and here for the same
|
|
151
181
|
* reason as the two above: it is a by-product recorded on the STEP, not the deliverable of any
|
|
@@ -357,6 +387,53 @@ const panelClass = computed(() => [
|
|
|
357
387
|
</div>
|
|
358
388
|
</section>
|
|
359
389
|
|
|
390
|
+
<!-- Shared trailing section: the bugfix reproduction proof (the declared check run against
|
|
391
|
+
the pre-fix tree and the final one). Collapsed when it proved the fix, expanded when it
|
|
392
|
+
did not — an inconclusive verdict or an infeasibility declaration is the one a reviewer
|
|
393
|
+
opened the window to read. -->
|
|
394
|
+
<section
|
|
395
|
+
v-if="reproductionReport"
|
|
396
|
+
class="shrink-0 border-t border-slate-800 bg-slate-900/60"
|
|
397
|
+
data-testid="result-window-reproduction"
|
|
398
|
+
>
|
|
399
|
+
<button
|
|
400
|
+
type="button"
|
|
401
|
+
class="flex w-full items-center gap-2 px-5 py-2 text-start hover:bg-slate-800/40"
|
|
402
|
+
:aria-expanded="reproductionOpen"
|
|
403
|
+
data-testid="result-window-reproduction-toggle"
|
|
404
|
+
@click="reproductionOpen = !reproductionOpen"
|
|
405
|
+
>
|
|
406
|
+
<UIcon
|
|
407
|
+
:name="reproductionKeys!.icon"
|
|
408
|
+
class="h-3.5 w-3.5 shrink-0"
|
|
409
|
+
:class="reproductionKeys!.proven ? 'text-emerald-400' : 'text-amber-400'"
|
|
410
|
+
/>
|
|
411
|
+
<span class="text-[11px] font-semibold uppercase tracking-wide text-slate-400">
|
|
412
|
+
{{ t('panels.stepDetail.reproduction.heading') }}
|
|
413
|
+
</span>
|
|
414
|
+
<span
|
|
415
|
+
class="shrink-0 rounded px-1.5 py-0.5 text-[11px] font-medium"
|
|
416
|
+
:class="
|
|
417
|
+
reproductionKeys!.proven
|
|
418
|
+
? 'bg-emerald-500/15 text-emerald-300'
|
|
419
|
+
: 'bg-amber-500/15 text-amber-300'
|
|
420
|
+
"
|
|
421
|
+
>
|
|
422
|
+
{{ t(reproductionKeys!.chip) }}
|
|
423
|
+
</span>
|
|
424
|
+
<span class="min-w-0 flex-1 truncate text-[12px] text-slate-400">
|
|
425
|
+
{{ t(reproductionKeys!.verdict) }}
|
|
426
|
+
</span>
|
|
427
|
+
<UIcon
|
|
428
|
+
:name="reproductionOpen ? 'i-lucide-chevron-down' : 'i-lucide-chevron-up'"
|
|
429
|
+
class="ms-auto h-3.5 w-3.5 shrink-0 text-slate-500"
|
|
430
|
+
/>
|
|
431
|
+
</button>
|
|
432
|
+
<div v-if="reproductionOpen" class="max-h-72 overflow-y-auto px-5 pb-3">
|
|
433
|
+
<StepReproductionReport :report="reproductionReport" />
|
|
434
|
+
</div>
|
|
435
|
+
</section>
|
|
436
|
+
|
|
360
437
|
<!-- Shared trailing section: what this step's agent declared it stored through a
|
|
361
438
|
foundational storage service (see `binaryOutputs` above for why it lives here and
|
|
362
439
|
not in a window). The collapsed row states the OUTCOME, never a count — five of
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
<script setup lang="ts">
|
|
2
|
+
import { computed } from 'vue'
|
|
3
|
+
import type { ReproductionPhaseOutcome, ReproductionReport } from '~/types/reproduction'
|
|
4
|
+
import { REPRODUCTION_STATUS_KEYS, REPRODUCTION_TREE_KEYS } from '~/utils/reproduction'
|
|
5
|
+
|
|
6
|
+
// The BUGFIX REPRODUCTION PROOF for a step: the declared reproducing check as the
|
|
7
|
+
// executor-harness ran it against the pre-fix tree and against the final tree, with both captured
|
|
8
|
+
// outputs. The verdict is computed from the two exit codes by the harness — never self-reported by
|
|
9
|
+
// the model, which is the entire point (the `repro-test` kind's own `outcome` has always been the
|
|
10
|
+
// agent's CLAIM, and this is what checks it). See
|
|
11
|
+
// backend/docs/adr/0033-bugfix-reproduction-proof.md.
|
|
12
|
+
//
|
|
13
|
+
// Two callers, one renderer, exactly as `StepEffortReport` has: the generic step-detail panel drops
|
|
14
|
+
// it in as a `card` (its own heading + border, among the other detail sections) and
|
|
15
|
+
// `ResultWindowShell`'s collapsible footer embeds it `flat`, where the disclosure row is already
|
|
16
|
+
// the heading. BOTH are needed, and that is the point: the engine records the proof on whichever
|
|
17
|
+
// step OPENED the pull request — in every built-in pipeline the `coder`, whose kind declares no
|
|
18
|
+
// result view and therefore opens the step-detail panel the shell is not involved in.
|
|
19
|
+
const props = withDefaults(
|
|
20
|
+
defineProps<{ report: ReproductionReport; variant?: 'card' | 'flat' }>(),
|
|
21
|
+
{ variant: 'flat' },
|
|
22
|
+
)
|
|
23
|
+
const { t } = useI18n()
|
|
24
|
+
|
|
25
|
+
/**
|
|
26
|
+
* The two trees, as rows. An ABSENT final run is normal rather than missing data: a green pre-fix
|
|
27
|
+
* tree already settles the verdict, so running the second one could only confirm what is already
|
|
28
|
+
* not proof — and each run costs a full setup plus test. The row says which, instead of leaving a
|
|
29
|
+
* blank a reader would read as a failure to record.
|
|
30
|
+
*/
|
|
31
|
+
const phases = computed(() => [
|
|
32
|
+
{ key: 'base' as const, label: REPRODUCTION_TREE_KEYS.base, outcome: props.report.base },
|
|
33
|
+
{ key: 'final' as const, label: REPRODUCTION_TREE_KEYS.final, outcome: props.report.final },
|
|
34
|
+
])
|
|
35
|
+
|
|
36
|
+
/** The verdict's copy + icon, from the exhaustive lookup rather than an assembled key. */
|
|
37
|
+
const presentation = computed(() => REPRODUCTION_STATUS_KEYS[props.report.status])
|
|
38
|
+
|
|
39
|
+
/** A phase's result chip: setup failure and timeout are their own answers, not plain failures. */
|
|
40
|
+
function phaseLabel(outcome: ReproductionPhaseOutcome): string {
|
|
41
|
+
if (outcome.setupFailed) return t('panels.stepDetail.reproduction.setupFailed')
|
|
42
|
+
if (outcome.timedOut) return t('panels.stepDetail.reproduction.timedOut')
|
|
43
|
+
return outcome.passed
|
|
44
|
+
? t('panels.stepDetail.reproduction.passed')
|
|
45
|
+
: t('panels.stepDetail.reproduction.exitCode', { code: outcome.exitCode })
|
|
46
|
+
}
|
|
47
|
+
</script>
|
|
48
|
+
|
|
49
|
+
<template>
|
|
50
|
+
<section
|
|
51
|
+
class="space-y-2"
|
|
52
|
+
data-testid="step-reproduction-report"
|
|
53
|
+
:class="
|
|
54
|
+
variant === 'card' ? 'scroll-mt-4 rounded-xl border border-slate-800 bg-slate-900/50 p-4' : ''
|
|
55
|
+
"
|
|
56
|
+
>
|
|
57
|
+
<div
|
|
58
|
+
v-if="variant === 'card'"
|
|
59
|
+
class="flex items-center gap-1.5 text-[11px] font-semibold uppercase tracking-wide text-slate-400"
|
|
60
|
+
>
|
|
61
|
+
<UIcon :name="presentation.icon" class="h-3.5 w-3.5" />
|
|
62
|
+
<span>{{ t('panels.stepDetail.reproduction.heading') }}</span>
|
|
63
|
+
</div>
|
|
64
|
+
|
|
65
|
+
<p class="text-[11px] text-slate-400" data-testid="reproduction-verdict">
|
|
66
|
+
{{ t(presentation.verdict) }}
|
|
67
|
+
</p>
|
|
68
|
+
|
|
69
|
+
<!-- The producer's own one-line diagnosis, rendered VERBATIM. Only the side that ran the two
|
|
70
|
+
trees can tell a test that misses the defect from a resumed run whose pre-fix tree already
|
|
71
|
+
carried this step's own interrupted work, so re-deriving a cause from the exit codes here
|
|
72
|
+
is exactly the inference that gets it wrong. -->
|
|
73
|
+
<p v-if="report.note" class="text-[11px] text-amber-300" data-testid="reproduction-observation">
|
|
74
|
+
{{ report.note }}
|
|
75
|
+
</p>
|
|
76
|
+
|
|
77
|
+
<!-- A structural infeasibility declaration: the reason, and what the agent verified instead.
|
|
78
|
+
This is what keeps "could not be reproduced" from looking like "nobody tried". -->
|
|
79
|
+
<template v-if="report.status === 'declared_infeasible'">
|
|
80
|
+
<div v-if="report.reason" class="rounded-md border border-slate-800 bg-slate-950/40 p-2">
|
|
81
|
+
<p class="text-[11px] font-semibold uppercase tracking-wide text-slate-400">
|
|
82
|
+
{{ t('panels.stepDetail.reproduction.reason') }}
|
|
83
|
+
</p>
|
|
84
|
+
<p class="mt-1 whitespace-pre-wrap text-[12px] text-slate-300">{{ report.reason }}</p>
|
|
85
|
+
</div>
|
|
86
|
+
<div
|
|
87
|
+
v-if="report.alternativeVerification"
|
|
88
|
+
class="rounded-md border border-slate-800 bg-slate-950/40 p-2"
|
|
89
|
+
>
|
|
90
|
+
<p class="text-[11px] font-semibold uppercase tracking-wide text-slate-400">
|
|
91
|
+
{{ t('panels.stepDetail.reproduction.alternative') }}
|
|
92
|
+
</p>
|
|
93
|
+
<p class="mt-1 whitespace-pre-wrap text-[12px] text-slate-300">
|
|
94
|
+
{{ report.alternativeVerification }}
|
|
95
|
+
</p>
|
|
96
|
+
</div>
|
|
97
|
+
</template>
|
|
98
|
+
|
|
99
|
+
<template v-else>
|
|
100
|
+
<div v-if="report.command" class="flex items-baseline gap-2">
|
|
101
|
+
<span class="shrink-0 text-[11px] text-slate-500">{{
|
|
102
|
+
t('panels.stepDetail.reproduction.command')
|
|
103
|
+
}}</span>
|
|
104
|
+
<span class="truncate font-mono text-[11px] text-slate-300">{{ report.command }}</span>
|
|
105
|
+
</div>
|
|
106
|
+
|
|
107
|
+
<!-- A dropped path can leave the pre-fix tree without the reproduction, which greens it and
|
|
108
|
+
reads as "the test does not capture the defect". Stated, never implied. -->
|
|
109
|
+
<p
|
|
110
|
+
v-if="report.omittedTestPaths"
|
|
111
|
+
class="text-[11px] text-amber-300"
|
|
112
|
+
data-testid="reproduction-omitted-paths"
|
|
113
|
+
>
|
|
114
|
+
{{
|
|
115
|
+
t(
|
|
116
|
+
'panels.stepDetail.reproduction.omittedTestPaths',
|
|
117
|
+
{ count: report.omittedTestPaths },
|
|
118
|
+
report.omittedTestPaths,
|
|
119
|
+
)
|
|
120
|
+
}}
|
|
121
|
+
</p>
|
|
122
|
+
|
|
123
|
+
<div
|
|
124
|
+
v-for="phase in phases"
|
|
125
|
+
:key="phase.key"
|
|
126
|
+
class="rounded-md border border-slate-800 bg-slate-950/40 p-2"
|
|
127
|
+
data-testid="reproduction-phase"
|
|
128
|
+
>
|
|
129
|
+
<div class="flex items-center gap-2">
|
|
130
|
+
<UIcon
|
|
131
|
+
:name="phase.outcome?.passed ? 'i-lucide-check' : 'i-lucide-x'"
|
|
132
|
+
class="h-3.5 w-3.5 shrink-0"
|
|
133
|
+
:class="
|
|
134
|
+
phase.outcome
|
|
135
|
+
? phase.outcome.passed
|
|
136
|
+
? 'text-emerald-400'
|
|
137
|
+
: 'text-rose-400'
|
|
138
|
+
: 'text-slate-600'
|
|
139
|
+
"
|
|
140
|
+
/>
|
|
141
|
+
<span class="text-[12px] font-medium text-slate-200">
|
|
142
|
+
{{ t(phase.label) }}
|
|
143
|
+
</span>
|
|
144
|
+
<span class="ms-auto shrink-0 text-[11px] tabular-nums text-slate-400">
|
|
145
|
+
{{
|
|
146
|
+
phase.outcome ? phaseLabel(phase.outcome) : t('panels.stepDetail.reproduction.notRun')
|
|
147
|
+
}}
|
|
148
|
+
</span>
|
|
149
|
+
</div>
|
|
150
|
+
<pre
|
|
151
|
+
v-if="phase.outcome?.outputTail"
|
|
152
|
+
class="mt-1.5 max-h-48 overflow-auto whitespace-pre-wrap break-words rounded bg-slate-950 p-2 font-mono text-[11px] text-slate-400"
|
|
153
|
+
data-testid="reproduction-output"
|
|
154
|
+
>{{ phase.outcome.outputTail }}</pre>
|
|
155
|
+
</div>
|
|
156
|
+
|
|
157
|
+
<p class="text-[11px] text-slate-500">
|
|
158
|
+
{{
|
|
159
|
+
t('panels.stepDetail.reproduction.attempts', {
|
|
160
|
+
attempts: report.attempts,
|
|
161
|
+
maxAttempts: report.maxAttempts,
|
|
162
|
+
})
|
|
163
|
+
}}
|
|
164
|
+
</p>
|
|
165
|
+
</template>
|
|
166
|
+
</section>
|
|
167
|
+
</template>
|