@pikku/core 0.12.115 → 0.12.118
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +81 -0
- package/dist/types/core.types.d.ts +2 -0
- package/dist/wirings/addon/wire-addon.d.ts +12 -0
- package/dist/wirings/addon/wire-remote-addon.d.ts +2 -0
- package/dist/wirings/channel/channel.types.d.ts +1 -0
- package/dist/wirings/mcp/mcp.types.d.ts +18 -0
- package/dist/wirings/workflow/pikku-scenario-service.d.ts +49 -3
- package/dist/wirings/workflow/pikku-scenario-service.js +95 -4
- package/dist/wirings/workflow/scenario-run.types.d.ts +65 -2
- package/dist/wirings/workflow/scenario-step.types.d.ts +21 -1
- package/dist/wirings/workflow/scenario.types.d.ts +11 -1
- package/package.json +1 -1
- package/src/public-surface.json +1 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,84 @@
|
|
|
1
|
+
## 0.12.118
|
|
2
|
+
|
|
3
|
+
### Patch Changes
|
|
4
|
+
|
|
5
|
+
- b4a895e: A scenario step row now carries where it fell inside each actor's recording, and a scenario result now says which feature it came from by id as well as by title.
|
|
6
|
+
|
|
7
|
+
The video offset is recorded rather than estimated. Documentation built out of a run has to turn a step sentence into an exact moment — a chapter marker, or a still pulled with `ffmpeg -ss` — and the only number available until now was the scenario clock summed out of the ladder. That clock is not the video's: recording starts when the actor's browser context opens, which is somewhere after step one, and every RPC step before or between the browser ones burns scenario time while the file sits still. The two drift apart by however much of the scenario happened off camera, and a still forty seconds out is a picture of the wrong screen with nothing to say it is wrong.
|
|
8
|
+
|
|
9
|
+
So `ScenarioStepRow.video` is stamped at the moment the step runs, from the driver's own clock: `ScenarioBrowserProvider.videoStartedAt(actor)` reports when that actor's context was opened with `recordVideo`, and the runner subtracts. It is a list of `{ actor, offsetMs }` rather than one number, because a video belongs to an actor and not to the scenario — one step touching two windows falls at a different moment in each, and an offset that does not name its file cannot be seeked to. A run without video, a step with no actor, and a step that never ran all carry nothing, which is what keeps the console's existing estimate as the fallback for runs recorded before this.
|
|
10
|
+
|
|
11
|
+
`ScenarioResult.featureId` is the other half of the same problem. `feature` is a title written for people to read and rewritten whenever the wording improves, so nothing downstream could key off it; the id `addFeature` registered the feature under does not move. The runner threads it from the plan, which read it off the registry, instead of deriving it from the label. `scenarioName` already carried the registration id and keeps it.
|
|
12
|
+
|
|
13
|
+
- b4a895e: `pikku scenario guide` writes the user guide a scenario suite already contains. A feature reads as a page and a scenario as a section, and a run leaves screenshots behind with the captions their author took them under — the command joins that to the editorial prose a project checks in under `docs/` and writes markdown. It renders no HTML, ships no components, resolves no asset URLs and calls no model: an image is an ordinary relative ``, and whoever consumes the markdown rewrites the paths.
|
|
14
|
+
|
|
15
|
+
A page cites a feature by leaving the marker pair where the block belongs:
|
|
16
|
+
|
|
17
|
+
```markdown
|
|
18
|
+
---
|
|
19
|
+
title: Deployments
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
A deployment is one tracked shipment of your app.
|
|
23
|
+
|
|
24
|
+
<!-- pikku:guide feature=deploymentsFeature -->
|
|
25
|
+
<!-- /pikku:guide -->
|
|
26
|
+
|
|
27
|
+
## Does my app go down during a deploy?
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
That one line does both halves of the job. It says _where_ the block goes, which a frontmatter list cannot express, and it is what the coverage gate counts to decide _whether_ a feature is documented at all. The mapping stays many-to-many and falls out of the union of every marker in the tree. A rebuild rewrites exactly the regions between the markers, so every sentence a human wrote around them survives.
|
|
31
|
+
|
|
32
|
+
**The steps are evidence, not content.** A generated block is the scenario's title, the description its author wrote, and the shots it filed — never a numbered Given/When/Then ladder, which is a test report and not something anybody arrives at a documentation page wanting. The sentences are still what the guide is kept honest against: `docs/.guide.lock` records a hash of each feature's step sentences and artifact ids, deliberately not of the image bytes. Restyling a UI changes every screenshot and no sentence; inserting or renaming a step changes what the prose was describing, and the page is reported stale. The lock is generated and checked in, so no hash is ever typed or merged by hand — and a tree whose lock is untracked reports every page as current forever.
|
|
33
|
+
|
|
34
|
+
Every registered feature has to be cited by some page, and a feature that is pure plumbing says so rather than being written about — `pikkuFeature({ document: false })`, threaded through the inspector and `FeatureMeta`. An uncited feature fails the command by name; `--allow-undocumented` downgrades that one failure to a report. A page citing a feature id that is not registered stays an error either way.
|
|
35
|
+
|
|
36
|
+
A guide is only written out of a run that can stand behind it. A run that failed or was killed halfway is refused, because a page is a claim that the product does what it says. So is a narrowed one: `pikku scenario run --flows`/`--features`/`--tags` leaves out scenarios the suite has, and a guide built from it would describe those flows as though they do not exist. `ScenarioRunRecord.selection` records the filters a run was selected with, since nothing in the results afterwards can tell a suite of forty from forty that were asked for.
|
|
37
|
+
|
|
38
|
+
Results are joined to features by `featureId`, falling back to the display name only for records written before that field existed — a title is rewritten freely and two features may share one.
|
|
39
|
+
|
|
40
|
+
Emission is deterministic — identical inputs give byte-identical output, and no timestamp goes in that did not come from the run record. Frontmatter the compiler does not own (`slug`, `draft`, `sidebar_position`, anything else a docs site reads) passes through untouched, and a source written with CRLF line endings is read as having frontmatter.
|
|
41
|
+
|
|
42
|
+
## 0.12.117
|
|
43
|
+
|
|
44
|
+
### Patch Changes
|
|
45
|
+
|
|
46
|
+
- 8a0ecb7: `pikku scenario run <env> --run browser --strict` now refuses every route off the run surface, rather than only reporting one at the end. An action step that would fall back to its `default` binding throws `ScenarioNoSurfaceBinding` even when it declares `default`, and a `then` witnessed only server-side throws the new `ScenarioUnwitnessedAssertion`. Without `--strict` nothing changes: the fallback still runs and the unwitnessed assertion is still counted into the coverage line.
|
|
47
|
+
|
|
48
|
+
It exists because a run is becoming a source of documentation. A docs build runs the suite `--run browser --strict`, so a feature that cannot be driven end to end through the UI cannot produce a page — no flow where three steps are screenshots and the fourth quietly happened over RPC with nothing to show. The refusal names the step, the surface the run asked for and the surfaces that did bind it, which turns "what still has no browser binding" into a worklist the run prints rather than something to go looking for.
|
|
49
|
+
|
|
50
|
+
`--strict` on `--run default` is accepted and does nothing, since nothing on the default surface can fall back or be witnessed elsewhere. `PikkuScenarioService.setRunSurface` takes the flag as a second argument, and `isStrictSurface()` reads it back.
|
|
51
|
+
|
|
52
|
+
## 0.12.116
|
|
53
|
+
|
|
54
|
+
### Patch Changes
|
|
55
|
+
|
|
56
|
+
- 87971bd: `browser.screenshot('the order, confirmed', { showcase: true })` marks one shot as fit to publish outside the run, and the artifact ledger carries the flag. A gallery, a docs page or a marketing card can then be built from the scenario run itself instead of a second browser pass configured somewhere else, and the author of the step — the only one who knows the page is at a moment worth showing a stranger — is who decides.
|
|
57
|
+
|
|
58
|
+
Each filed screenshot also carries an `id`: the same shot under one key across runs. `path` leads with the order the run happened in, so inserting a step ahead of a shot renumbers it and anything meant to outlive one run (a caption override, a diff against last week's build) loses track of it.
|
|
59
|
+
|
|
60
|
+
Two supporting fixes in `@pikku/playwright`: contexts open at a pinned `viewport` (1440x900, overridable per config or via `E2E_VIEWPORT_WIDTH`/`E2E_VIEWPORT_HEIGHT`) and screenshots are taken with animations disabled, so two runs of the same scenario photograph the same thing. `{ fullPage: true }` is available for shots of a whole scrollable page.
|
|
61
|
+
|
|
62
|
+
- b312867: A project can now serve several MCP endpoints, one per connector.
|
|
63
|
+
|
|
64
|
+
Until now every MCP tool in a project was pooled onto a single `/mcp`, so a hub offering three connectors offered one endpoint listing all three connectors' tools at once. A client pointed at it saw tools it had no business calling, and the only way to give a connector an endpoint of its own was to give it a deployment of its own — three deploys, three bills, three service graphs.
|
|
65
|
+
|
|
66
|
+
`wireAddon` gains `mcpEndpoint`. `true` serves that instance's tools at `/mcp/<name>`; a string is the path, used as given. Leaving it unset keeps the tools on the shared endpoint, which is where they have always been, so nothing existing moves.
|
|
67
|
+
|
|
68
|
+
A surfaced instance now gets its own manifest (`.pikku/mcp/mcp.<name>.gen.json`, carrying the path it answers on), its own deploy unit (`mcp-<name>`, routed on that path), and its own MCP server — with its own tool list, so a client pointed at one endpoint never sees another's tools. The plumbing for the per-surface manifest and unit already existed in `deploy apply`; nothing had ever produced one.
|
|
69
|
+
|
|
70
|
+
`pikku dev` mounts every endpoint the generated tree describes, not just the default one. Without that a project that moved its tools onto their own endpoints would have served nothing locally at all — the default manifest it reads is empty precisely because they moved — and the only way to try a connector would have been to deploy it.
|
|
71
|
+
|
|
72
|
+
The node and bun transports take `mcpSurfaces` alongside `mcpJson` and mount each at its own path, longest path first. `/mcp` claims everything beneath `/mcp/`, so without that ordering the default endpoint answers `/mcp/weather` and the surface's tools are unreachable.
|
|
73
|
+
|
|
74
|
+
OAuth discovery is split between the endpoints rather than duplicated across them. RFC 9728 folds a resource's path into its well-known route, so each endpoint's own document is already distinct, but the path-less `/.well-known/oauth-protected-resource` predates that and describes whichever resource answers it. Only the default endpoint claims it — otherwise every unit registers the same route and the provider's router decides which resource a client is told about, and in dev a client probing it is described whichever surface sorted first.
|
|
75
|
+
|
|
76
|
+
- 51bd35a: Document five public keys that carried no JSDoc: `wireChannel`'s `onDisconnect`,
|
|
77
|
+
`wireRemoteAddon`'s `serverUrl` and `tags`, and `CoreUserSession`'s `userId` and
|
|
78
|
+
`orgId`. A key printed as a name and a type is a shape; what a caller needs is
|
|
79
|
+
what to put in it, and only the JSDoc where the type is declared carries that
|
|
80
|
+
into the IDE, the console and the shipped surface doc at once.
|
|
81
|
+
|
|
1
82
|
## 0.12.115
|
|
2
83
|
|
|
3
84
|
### Patch Changes
|
|
@@ -81,7 +81,9 @@ export type CoreConfig<Config extends Record<string, unknown> = {}> = {
|
|
|
81
81
|
postgres?: PostgresConfig;
|
|
82
82
|
} & Config;
|
|
83
83
|
export interface CoreUserSession {
|
|
84
|
+
/** Who the session belongs to, as your own system identifies them. Pikku only carries it. */
|
|
84
85
|
userId?: string;
|
|
86
|
+
/** The tenant the session is acting inside, when the project has more than one. */
|
|
85
87
|
orgId?: string;
|
|
86
88
|
/** True when the session belongs to a synthetic scenario actor — lets audits/analytics address synthetic traffic */
|
|
87
89
|
actor?: boolean;
|
|
@@ -16,6 +16,18 @@ export type WireAddonConfig = {
|
|
|
16
16
|
* and is typed against the addon's function names.
|
|
17
17
|
*/
|
|
18
18
|
mcp?: boolean | string[];
|
|
19
|
+
/**
|
|
20
|
+
* Serves this addon's MCP tools on an endpoint of their own rather than
|
|
21
|
+
* folding them into the project's single `/mcp`. `true` mounts them at
|
|
22
|
+
* `/mcp/<name>`; a string is the path, used as given.
|
|
23
|
+
*
|
|
24
|
+
* This is what lets one project expose several connectors: each wired
|
|
25
|
+
* instance becomes its own MCP server, with its own tool list, so a client
|
|
26
|
+
* pointed at one never sees another's tools. Leaving it unset keeps the
|
|
27
|
+
* addon's tools on the shared endpoint, which is where they have always
|
|
28
|
+
* been.
|
|
29
|
+
*/
|
|
30
|
+
mcpEndpoint?: boolean | string;
|
|
19
31
|
/** Filters this addon in and out of a build — see the `tags` option on `pikku all`. It has no effect at runtime. */
|
|
20
32
|
tags?: string[];
|
|
21
33
|
/** Required of every function in the addon, on top of the function's own. */
|
|
@@ -12,11 +12,13 @@ export type WireRemoteAddonConfig = {
|
|
|
12
12
|
name: string;
|
|
13
13
|
/** Must be installed as a devDependency — `pikku verify` enforces this. */
|
|
14
14
|
package: string;
|
|
15
|
+
/** Where the addon is deployed, e.g. `https://registry.example.com`. A function when it varies per environment. */
|
|
15
16
|
serverUrl: string | ((services: CoreServices) => string | Promise<string>);
|
|
16
17
|
/** Omit when the addon declares its remote surface public. */
|
|
17
18
|
auth?: RemoteAddonAuth;
|
|
18
19
|
/** Map a consumer-facing fn name → the remote fn name, when they differ (rare). */
|
|
19
20
|
remoteName?: (fn: string) => string;
|
|
21
|
+
/** Applied to every function the addon contributes, so tag middleware and permissions reach them. */
|
|
20
22
|
tags?: string[];
|
|
21
23
|
};
|
|
22
24
|
/**
|
|
@@ -68,6 +68,7 @@ export type CoreChannel<ChannelData, Channel extends string, ChannelConnect = Co
|
|
|
68
68
|
func?: ChannelConnect;
|
|
69
69
|
middleware?: PikkuMiddleware[];
|
|
70
70
|
};
|
|
71
|
+
/** Runs once after the socket closes, however it closed. Nothing it returns reaches the client. */
|
|
71
72
|
onDisconnect?: ChannelDisconnect | {
|
|
72
73
|
func?: ChannelDisconnect;
|
|
73
74
|
middleware?: PikkuMiddleware[];
|
|
@@ -21,12 +21,24 @@ export type MCPResourceMeta = Record<string, Omit<CoreMCPResource, 'func' | 'mid
|
|
|
21
21
|
inputSchema: string | null;
|
|
22
22
|
outputSchema: string | null;
|
|
23
23
|
middleware?: MiddlewareMetadata[];
|
|
24
|
+
/**
|
|
25
|
+
* The MCP endpoint this belongs to, when the project serves more than one.
|
|
26
|
+
* Absent means the project's default endpoint. Codegen splits the metadata
|
|
27
|
+
* by this key, so each endpoint's manifest lists only its own.
|
|
28
|
+
*/
|
|
29
|
+
surface?: string;
|
|
24
30
|
}>;
|
|
25
31
|
export type MCPToolMeta = Record<string, Omit<CoreMCPTool, 'func' | 'middleware'> & {
|
|
26
32
|
pikkuFuncId: string;
|
|
27
33
|
inputSchema: string | null;
|
|
28
34
|
outputSchema: string | null;
|
|
29
35
|
middleware?: MiddlewareMetadata[];
|
|
36
|
+
/**
|
|
37
|
+
* The MCP endpoint this belongs to, when the project serves more than one.
|
|
38
|
+
* Absent means the project's default endpoint. Codegen splits the metadata
|
|
39
|
+
* by this key, so each endpoint's manifest lists only its own.
|
|
40
|
+
*/
|
|
41
|
+
surface?: string;
|
|
30
42
|
}>;
|
|
31
43
|
export type MCPPromptMeta = Record<string, Omit<CoreMCPPrompt, 'func' | 'middleware'> & {
|
|
32
44
|
pikkuFuncId: string;
|
|
@@ -38,6 +50,12 @@ export type MCPPromptMeta = Record<string, Omit<CoreMCPPrompt, 'func' | 'middlew
|
|
|
38
50
|
required: boolean;
|
|
39
51
|
}>;
|
|
40
52
|
middleware?: MiddlewareMetadata[];
|
|
53
|
+
/**
|
|
54
|
+
* The MCP endpoint this belongs to, when the project serves more than one.
|
|
55
|
+
* Absent means the project's default endpoint. Codegen splits the metadata
|
|
56
|
+
* by this key, so each endpoint's manifest lists only its own.
|
|
57
|
+
*/
|
|
58
|
+
surface?: string;
|
|
41
59
|
}>;
|
|
42
60
|
export type CoreMCPResource<PikkuFunctionConfig = CorePikkuFunctionConfig<CorePikkuFunctionSessionless<any, any>>, PikkuPermission = CorePikkuPermission<any, any>, PikkuMiddleware = CorePikkuMiddleware<any>> = {
|
|
43
61
|
/** How the client addresses this resource. `{name}` marks a parameter, and every parameter must be a key of the function's input schema. */
|
|
@@ -3,11 +3,12 @@ import { InMemoryWorkflowService } from '../../services/in-memory-workflow-servi
|
|
|
3
3
|
import type { RunLifecycleContext, WorkflowRunEngine, WorkflowRunExtension } from './workflow-run-engine.types.js';
|
|
4
4
|
import type { PikkuRawWire } from '../../types/core.types.js';
|
|
5
5
|
import type { ScenarioPersonas } from '../../services/personas-service.js';
|
|
6
|
+
import type { ScenarioStepVideoOffset } from './scenario-run.types.js';
|
|
6
7
|
import type { ScenarioBrowserProvider, ScenarioEnvironment, ScenarioSurface } from './scenario-step.types.js';
|
|
7
8
|
import type { PikkuWorkflowWire, WorkflowQueueOptions } from './workflow.types.js';
|
|
8
9
|
export { addFeature, resolveFeatureScenarios } from './feature.js';
|
|
9
|
-
export type { CoreFeature, CoreFeatureScenario, FeatureMeta, FeaturesMeta, PikkuBrowserWire, PikkuScenarioWire, ScenarioBrowserFailure, ScenarioBrowserProvider, ScenarioEnvironment, ScenarioStepKind, ScenarioStepMeta, ScenarioStepOptions, ScenarioStepPhase, ScenarioSurface, TestIdSelector, } from './scenario.types.js';
|
|
10
|
-
export type { ScenarioArtifact, ScenarioFailureDetail, ScenarioResult, ScenarioRunRecord, ScenarioRunReport, ScenarioRunStatus, ScenarioRunStore, ScenarioRunSummary, ScenarioStepRow, } from './scenario-run.types.js';
|
|
10
|
+
export type { CoreFeature, CoreFeatureScenario, FeatureMeta, FeaturesMeta, PikkuBrowserWire, PikkuScenarioWire, ScenarioBrowserFailure, ScenarioBrowserProvider, ScenarioEnvironment, ScenarioStepKind, ScenarioStepMeta, ScenarioStepOptions, ScenarioScreenshotOptions, ScenarioStepPhase, ScenarioSurface, TestIdSelector, } from './scenario.types.js';
|
|
11
|
+
export type { ScenarioArtifact, ScenarioFailureDetail, ScenarioResult, ScenarioRunRecord, ScenarioRunReport, ScenarioRunSelection, ScenarioRunStatus, ScenarioRunStore, ScenarioRunSummary, ScenarioStepRow, ScenarioStepVideoOffset, } from './scenario-run.types.js';
|
|
11
12
|
export { SCENARIO_SURFACES } from './scenario-step.types.js';
|
|
12
13
|
export { resolveScenarioSurfaces } from './scenario-surface.js';
|
|
13
14
|
export { pollUntil, type PollOptions } from './scenario-poll.js';
|
|
@@ -92,6 +93,9 @@ export declare class ScenarioActorRequired extends PikkuError {
|
|
|
92
93
|
* coverage gap. (A `then` that ran somewhere, just not on the run surface, is
|
|
93
94
|
* the coverage gap; it is reported rather than thrown. See
|
|
94
95
|
* {@link ScenarioNoWitness} for one that ran nowhere.)
|
|
96
|
+
*
|
|
97
|
+
* A strict run also throws it for a step that *could* fall back, because there
|
|
98
|
+
* the fallback is the thing being refused.
|
|
95
99
|
*/
|
|
96
100
|
export declare class ScenarioNoSurfaceBinding extends PikkuError {
|
|
97
101
|
readonly stepFunc: string;
|
|
@@ -114,6 +118,22 @@ export declare class ScenarioNoWitness extends PikkuError {
|
|
|
114
118
|
readonly runSurface: ScenarioSurface;
|
|
115
119
|
constructor(stepFunc: string, declared: ScenarioSurface[], runSurface: ScenarioSurface);
|
|
116
120
|
}
|
|
121
|
+
/**
|
|
122
|
+
* An assertion ran, but not on the surface the run targets.
|
|
123
|
+
*
|
|
124
|
+
* The sibling of {@link ScenarioNoWitness}: that one checked nothing anywhere,
|
|
125
|
+
* this one checked the system of record while the prose claims the actor saw it
|
|
126
|
+
* on the page. Ordinary runs count it as a coverage gap; a strict run refuses
|
|
127
|
+
* it, because a flow that cannot be observed end to end on the run surface is
|
|
128
|
+
* not a flow that surface can be documented from.
|
|
129
|
+
*/
|
|
130
|
+
export declare class ScenarioUnwitnessedAssertion extends PikkuError {
|
|
131
|
+
readonly stepFunc: string;
|
|
132
|
+
readonly declared: ScenarioSurface[];
|
|
133
|
+
readonly runSurface: ScenarioSurface;
|
|
134
|
+
readonly witnessedOn: ScenarioSurface[];
|
|
135
|
+
constructor(stepFunc: string, declared: ScenarioSurface[], runSurface: ScenarioSurface, witnessedOn: ScenarioSurface[]);
|
|
136
|
+
}
|
|
117
137
|
/**
|
|
118
138
|
* The scenario capability, layered onto a workflow service rather than being
|
|
119
139
|
* one.
|
|
@@ -142,16 +162,25 @@ export declare class PikkuScenarioService implements WorkflowRunExtension {
|
|
|
142
162
|
private readonly engine;
|
|
143
163
|
private runActors;
|
|
144
164
|
private runContexts;
|
|
165
|
+
private runVideoOffsets;
|
|
145
166
|
private scenarioBrowserProvider?;
|
|
146
167
|
private scenarioEnvironment?;
|
|
147
168
|
private runSurface;
|
|
169
|
+
private strictSurface;
|
|
148
170
|
constructor(engine: WorkflowRunEngine);
|
|
149
171
|
/**
|
|
150
172
|
* The surface every actor drives the system through for this run, set once by
|
|
151
173
|
* the runner from `--run`. `default` is the server-side path — the fast suite.
|
|
174
|
+
*
|
|
175
|
+
* `strict` removes both routes off that surface: an action may not fall back
|
|
176
|
+
* to its `default` binding, and a `then` may not be witnessed anywhere else.
|
|
177
|
+
* It is what lets a run stand as evidence of the whole flow on one surface,
|
|
178
|
+
* which is what generating documentation from a run requires. On a `default`
|
|
179
|
+
* run it changes nothing, because nothing there can fall back.
|
|
152
180
|
*/
|
|
153
|
-
setRunSurface(surface: ScenarioSurface): void;
|
|
181
|
+
setRunSurface(surface: ScenarioSurface, strict?: boolean): void;
|
|
154
182
|
getRunSurface(): ScenarioSurface;
|
|
183
|
+
isStrictSurface(): boolean;
|
|
155
184
|
/**
|
|
156
185
|
* Registered by `@pikku/playwright` (or any other driver) before a scenario
|
|
157
186
|
* runs. Absent means browser steps cannot run, which the CLI checks up front
|
|
@@ -166,6 +195,23 @@ export declare class PikkuScenarioService implements WorkflowRunExtension {
|
|
|
166
195
|
*/
|
|
167
196
|
setScenarioEnvironment(env: ScenarioEnvironment | undefined): void;
|
|
168
197
|
getScenarioEnvironment(): ScenarioEnvironment | undefined;
|
|
198
|
+
/**
|
|
199
|
+
* Where each of a run's browser steps fell in its actor's video, keyed by the
|
|
200
|
+
* durable step name, handed over and forgotten in one call.
|
|
201
|
+
*
|
|
202
|
+
* Taken rather than read because the runner joins these onto the step rows
|
|
203
|
+
* after the run has finished — which is past the point anything else would
|
|
204
|
+
* clear them, and the only moment they are still wanted.
|
|
205
|
+
*/
|
|
206
|
+
takeStepVideoOffsets(runId: string): Map<string, ScenarioStepVideoOffset[]>;
|
|
207
|
+
/**
|
|
208
|
+
* Stamp where a step began inside one actor's recording.
|
|
209
|
+
*
|
|
210
|
+
* First write per actor wins: a `then` step runs once per witness, and the
|
|
211
|
+
* moment the reader wants is when the sentence started, not when its last
|
|
212
|
+
* witness got around to the browser.
|
|
213
|
+
*/
|
|
214
|
+
private recordVideoOffset;
|
|
169
215
|
attachRunContext(runId: string, workflowMeta: any, options?: {
|
|
170
216
|
actors?: ScenarioPersonas;
|
|
171
217
|
}): Promise<void>;
|
|
@@ -132,14 +132,21 @@ addError(ScenarioActorRequired, {
|
|
|
132
132
|
* coverage gap. (A `then` that ran somewhere, just not on the run surface, is
|
|
133
133
|
* the coverage gap; it is reported rather than thrown. See
|
|
134
134
|
* {@link ScenarioNoWitness} for one that ran nowhere.)
|
|
135
|
+
*
|
|
136
|
+
* A strict run also throws it for a step that *could* fall back, because there
|
|
137
|
+
* the fallback is the thing being refused.
|
|
135
138
|
*/
|
|
136
139
|
export class ScenarioNoSurfaceBinding extends PikkuError {
|
|
137
140
|
stepFunc;
|
|
138
141
|
declared;
|
|
139
142
|
runSurface;
|
|
140
143
|
constructor(stepFunc, declared, runSurface) {
|
|
141
|
-
|
|
142
|
-
|
|
144
|
+
const declares = `(declares: ${declared.join(', ') || 'nothing'})`;
|
|
145
|
+
super(declared.includes('default')
|
|
146
|
+
? `[scenario] step '${stepFunc}' has no binding for '${runSurface}' and a strict run will not let it ` +
|
|
147
|
+
`fall back to 'default' ${declares}. Add a '${runSurface}' binding, or drop --strict.`
|
|
148
|
+
: `[scenario] step '${stepFunc}' declares no binding for '${runSurface}' and no 'default' to fall back to ` +
|
|
149
|
+
`${declares}.`);
|
|
143
150
|
this.stepFunc = stepFunc;
|
|
144
151
|
this.declared = declared;
|
|
145
152
|
this.runSurface = runSurface;
|
|
@@ -175,6 +182,34 @@ addError(ScenarioNoWitness, {
|
|
|
175
182
|
status: 500,
|
|
176
183
|
message: 'Assertion has no witness for this surface.',
|
|
177
184
|
});
|
|
185
|
+
/**
|
|
186
|
+
* An assertion ran, but not on the surface the run targets.
|
|
187
|
+
*
|
|
188
|
+
* The sibling of {@link ScenarioNoWitness}: that one checked nothing anywhere,
|
|
189
|
+
* this one checked the system of record while the prose claims the actor saw it
|
|
190
|
+
* on the page. Ordinary runs count it as a coverage gap; a strict run refuses
|
|
191
|
+
* it, because a flow that cannot be observed end to end on the run surface is
|
|
192
|
+
* not a flow that surface can be documented from.
|
|
193
|
+
*/
|
|
194
|
+
export class ScenarioUnwitnessedAssertion extends PikkuError {
|
|
195
|
+
stepFunc;
|
|
196
|
+
declared;
|
|
197
|
+
runSurface;
|
|
198
|
+
witnessedOn;
|
|
199
|
+
constructor(stepFunc, declared, runSurface, witnessedOn) {
|
|
200
|
+
super(`[scenario] assertion '${stepFunc}' was checked on ${witnessedOn.join(', ')}, not on '${runSurface}'. ` +
|
|
201
|
+
`It held there — but nothing looked at '${runSurface}', which is what the step's prose claims the actor saw. ` +
|
|
202
|
+
`Add a '${runSurface}' witness (declares: ${declared.join(', ') || 'nothing'}), or drop --strict.`);
|
|
203
|
+
this.stepFunc = stepFunc;
|
|
204
|
+
this.declared = declared;
|
|
205
|
+
this.runSurface = runSurface;
|
|
206
|
+
this.witnessedOn = witnessedOn;
|
|
207
|
+
}
|
|
208
|
+
}
|
|
209
|
+
addError(ScenarioUnwitnessedAssertion, {
|
|
210
|
+
status: 500,
|
|
211
|
+
message: 'Assertion was checked, but not on the run surface.',
|
|
212
|
+
});
|
|
178
213
|
/**
|
|
179
214
|
* The scenario capability, layered onto a workflow service rather than being
|
|
180
215
|
* one.
|
|
@@ -208,22 +243,37 @@ export class PikkuScenarioService {
|
|
|
208
243
|
// so the body and its hooks share one object rather than reading it back off
|
|
209
244
|
// the persisted wire.
|
|
210
245
|
runContexts = new Map();
|
|
246
|
+
// Where each browser step landed in its actor's recording, per run. Held
|
|
247
|
+
// apart from the run context because the runner reads it once the run has
|
|
248
|
+
// ended, which is exactly when `detachRunContext` has cleared that.
|
|
249
|
+
runVideoOffsets = new Map();
|
|
211
250
|
scenarioBrowserProvider;
|
|
212
251
|
scenarioEnvironment;
|
|
213
252
|
runSurface = 'default';
|
|
253
|
+
strictSurface = false;
|
|
214
254
|
constructor(engine) {
|
|
215
255
|
this.engine = engine;
|
|
216
256
|
}
|
|
217
257
|
/**
|
|
218
258
|
* The surface every actor drives the system through for this run, set once by
|
|
219
259
|
* the runner from `--run`. `default` is the server-side path — the fast suite.
|
|
260
|
+
*
|
|
261
|
+
* `strict` removes both routes off that surface: an action may not fall back
|
|
262
|
+
* to its `default` binding, and a `then` may not be witnessed anywhere else.
|
|
263
|
+
* It is what lets a run stand as evidence of the whole flow on one surface,
|
|
264
|
+
* which is what generating documentation from a run requires. On a `default`
|
|
265
|
+
* run it changes nothing, because nothing there can fall back.
|
|
220
266
|
*/
|
|
221
|
-
setRunSurface(surface) {
|
|
267
|
+
setRunSurface(surface, strict = false) {
|
|
222
268
|
this.runSurface = surface;
|
|
269
|
+
this.strictSurface = strict;
|
|
223
270
|
}
|
|
224
271
|
getRunSurface() {
|
|
225
272
|
return this.runSurface;
|
|
226
273
|
}
|
|
274
|
+
isStrictSurface() {
|
|
275
|
+
return this.strictSurface;
|
|
276
|
+
}
|
|
227
277
|
/**
|
|
228
278
|
* Registered by `@pikku/playwright` (or any other driver) before a scenario
|
|
229
279
|
* runs. Absent means browser steps cannot run, which the CLI checks up front
|
|
@@ -246,6 +296,39 @@ export class PikkuScenarioService {
|
|
|
246
296
|
getScenarioEnvironment() {
|
|
247
297
|
return this.scenarioEnvironment;
|
|
248
298
|
}
|
|
299
|
+
/**
|
|
300
|
+
* Where each of a run's browser steps fell in its actor's video, keyed by the
|
|
301
|
+
* durable step name, handed over and forgotten in one call.
|
|
302
|
+
*
|
|
303
|
+
* Taken rather than read because the runner joins these onto the step rows
|
|
304
|
+
* after the run has finished — which is past the point anything else would
|
|
305
|
+
* clear them, and the only moment they are still wanted.
|
|
306
|
+
*/
|
|
307
|
+
takeStepVideoOffsets(runId) {
|
|
308
|
+
const offsets = this.runVideoOffsets.get(runId);
|
|
309
|
+
this.runVideoOffsets.delete(runId);
|
|
310
|
+
return offsets ?? new Map();
|
|
311
|
+
}
|
|
312
|
+
/**
|
|
313
|
+
* Stamp where a step began inside one actor's recording.
|
|
314
|
+
*
|
|
315
|
+
* First write per actor wins: a `then` step runs once per witness, and the
|
|
316
|
+
* moment the reader wants is when the sentence started, not when its last
|
|
317
|
+
* witness got around to the browser.
|
|
318
|
+
*/
|
|
319
|
+
recordVideoOffset(runId, stepName, actor, offsetMs) {
|
|
320
|
+
let byStep = this.runVideoOffsets.get(runId);
|
|
321
|
+
if (!byStep) {
|
|
322
|
+
byStep = new Map();
|
|
323
|
+
this.runVideoOffsets.set(runId, byStep);
|
|
324
|
+
}
|
|
325
|
+
const offsets = byStep.get(stepName) ?? [];
|
|
326
|
+
if (offsets.some((offset) => offset.actor === actor)) {
|
|
327
|
+
return;
|
|
328
|
+
}
|
|
329
|
+
offsets.push({ actor, offsetMs });
|
|
330
|
+
byStep.set(stepName, offsets);
|
|
331
|
+
}
|
|
249
332
|
async attachRunContext(runId, workflowMeta, options) {
|
|
250
333
|
const actors = options?.actors ??
|
|
251
334
|
(workflowMeta.source === 'scenario'
|
|
@@ -595,6 +678,10 @@ export class PikkuScenarioService {
|
|
|
595
678
|
// The dispatch guard above already refused an actor-less call, and a
|
|
596
679
|
// browser binding always requires one.
|
|
597
680
|
wire.browser = await this.scenarioBrowserProvider.sessionFor(actor.name);
|
|
681
|
+
const videoStartedAt = this.scenarioBrowserProvider.videoStartedAt?.(actor.name);
|
|
682
|
+
if (videoStartedAt !== undefined) {
|
|
683
|
+
this.recordVideoOffset(runId, stepName, actor.name, Math.max(0, Date.now() - videoStartedAt));
|
|
684
|
+
}
|
|
598
685
|
}
|
|
599
686
|
return await runPikkuFunc('workflow', workflowName, resolvedStepFunc, {
|
|
600
687
|
singletonServices: getSingletonServices(),
|
|
@@ -605,7 +692,8 @@ export class PikkuScenarioService {
|
|
|
605
692
|
});
|
|
606
693
|
};
|
|
607
694
|
if (resolution.kind === 'action') {
|
|
608
|
-
if (resolution.fellBack &&
|
|
695
|
+
if (resolution.fellBack &&
|
|
696
|
+
(this.strictSurface || !declared.includes('default'))) {
|
|
609
697
|
throw new ScenarioNoSurfaceBinding(resolvedStepFunc, declared, this.runSurface);
|
|
610
698
|
}
|
|
611
699
|
return await runOnSurface(resolution.surface);
|
|
@@ -613,6 +701,9 @@ export class PikkuScenarioService {
|
|
|
613
701
|
if (resolution.surfaces.length === 0) {
|
|
614
702
|
throw new ScenarioNoWitness(resolvedStepFunc, declared, this.runSurface);
|
|
615
703
|
}
|
|
704
|
+
if (this.strictSurface && resolution.unwitnessed) {
|
|
705
|
+
throw new ScenarioUnwitnessedAssertion(resolvedStepFunc, declared, this.runSurface, resolution.surfaces);
|
|
706
|
+
}
|
|
616
707
|
// A `then` runs every witness it has and they must agree. The surface
|
|
617
708
|
// witness runs first so that when the page is the thing that is wrong,
|
|
618
709
|
// it is the failure that surfaces.
|
|
@@ -31,6 +31,30 @@ export interface ScenarioArtifact {
|
|
|
31
31
|
actor?: string;
|
|
32
32
|
/** The caption the scenario author took a screenshot under. */
|
|
33
33
|
name?: string;
|
|
34
|
+
/**
|
|
35
|
+
* The same shot across runs, under one key. `path` carries the order the run
|
|
36
|
+
* happened in, so it moves whenever a step is inserted before it — nothing
|
|
37
|
+
* that outlives a run (a caption override, a diff against last week) can key
|
|
38
|
+
* off it.
|
|
39
|
+
*/
|
|
40
|
+
id?: string;
|
|
41
|
+
/** Fit to show outside the run: a marketing card, a docs page, a gallery. */
|
|
42
|
+
showcase?: boolean;
|
|
43
|
+
}
|
|
44
|
+
/**
|
|
45
|
+
* Where a step falls inside one actor's recording.
|
|
46
|
+
*
|
|
47
|
+
* Measured from the moment that actor's browser context opened — which is when
|
|
48
|
+
* Playwright starts the file — rather than from the start of the scenario, so
|
|
49
|
+
* it addresses the video's own clock. The two differ by however long the
|
|
50
|
+
* scenario spent before that window existed, plus every non-browser step since,
|
|
51
|
+
* which is why the offset is recorded at the moment the step runs instead of
|
|
52
|
+
* being summed back out of the ladder afterwards.
|
|
53
|
+
*/
|
|
54
|
+
export interface ScenarioStepVideoOffset {
|
|
55
|
+
/** Whose recording this offset is into: one actor, one video file. */
|
|
56
|
+
actor: string;
|
|
57
|
+
offsetMs: number;
|
|
34
58
|
}
|
|
35
59
|
/** One step of a run, already joined to the prose that declared it. */
|
|
36
60
|
export interface ScenarioStepRow {
|
|
@@ -45,6 +69,16 @@ export interface ScenarioStepRow {
|
|
|
45
69
|
status: string;
|
|
46
70
|
durationMs?: number;
|
|
47
71
|
error?: string;
|
|
72
|
+
/**
|
|
73
|
+
* Where this step lands in each actor's video, for the actors whose window
|
|
74
|
+
* was being recorded when it ran. Absent for a step with no actor, a run
|
|
75
|
+
* without video, and a step that never ran at all.
|
|
76
|
+
*
|
|
77
|
+
* A list rather than one number because a video belongs to an actor, not to
|
|
78
|
+
* the scenario: a step touching two windows falls at a different moment in
|
|
79
|
+
* each, and an offset that does not name its file cannot be seeked to.
|
|
80
|
+
*/
|
|
81
|
+
video?: ScenarioStepVideoOffset[];
|
|
48
82
|
}
|
|
49
83
|
/** Everything known about why one scenario failed. */
|
|
50
84
|
export interface ScenarioFailureDetail {
|
|
@@ -67,10 +101,22 @@ export interface ScenarioResult {
|
|
|
67
101
|
error?: string;
|
|
68
102
|
steps?: ScenarioStepRow[];
|
|
69
103
|
failure?: ScenarioFailureDetail;
|
|
70
|
-
/**
|
|
104
|
+
/**
|
|
105
|
+
* The registered scenario this ran, by the id it is registered under — the
|
|
106
|
+
* same id `FeatureMetaEntry.scenario` references. The label alone does not
|
|
107
|
+
* give it, and unlike the label it is not rewritten when the prose is.
|
|
108
|
+
*/
|
|
71
109
|
scenarioName?: string;
|
|
72
|
-
/** The feature that grouped it,
|
|
110
|
+
/** The feature that grouped it — its display name, which is freely renamed. */
|
|
73
111
|
feature?: string;
|
|
112
|
+
/**
|
|
113
|
+
* The registered feature that grouped it, by id.
|
|
114
|
+
*
|
|
115
|
+
* `feature` is a title someone writes for people to read, so nothing that
|
|
116
|
+
* outlives a run can key off it. This is what `addFeature` registered the
|
|
117
|
+
* feature under, and it is what survives the title being rewritten.
|
|
118
|
+
*/
|
|
119
|
+
featureId?: string;
|
|
74
120
|
tags?: string[];
|
|
75
121
|
/** Images and footage this scenario produced, filed under the run. */
|
|
76
122
|
artifacts?: ScenarioArtifact[];
|
|
@@ -94,6 +140,21 @@ export interface ScenarioRunReport {
|
|
|
94
140
|
hookFailures: string[];
|
|
95
141
|
}
|
|
96
142
|
export type ScenarioRunStatus = 'running' | 'passed' | 'failed';
|
|
143
|
+
/**
|
|
144
|
+
* The filters a run was selected with, recorded when it was narrowed at all.
|
|
145
|
+
*
|
|
146
|
+
* A narrowed run is a partial record of the suite: scenarios a feature owns can
|
|
147
|
+
* be missing from it, and whole features can be absent, with nothing in the
|
|
148
|
+
* results to say so. Anything that reads a run as evidence of what the suite
|
|
149
|
+
* does — rather than of what happened that afternoon — has to be able to tell
|
|
150
|
+
* the two apart, and it cannot be inferred from the results afterwards.
|
|
151
|
+
*/
|
|
152
|
+
export interface ScenarioRunSelection {
|
|
153
|
+
flows?: string[];
|
|
154
|
+
features?: string[];
|
|
155
|
+
tags?: string[];
|
|
156
|
+
excludeTags?: string[];
|
|
157
|
+
}
|
|
97
158
|
/**
|
|
98
159
|
* A whole run, as it is stored and read back.
|
|
99
160
|
*
|
|
@@ -107,6 +168,8 @@ export interface ScenarioRunRecord extends ScenarioRunReport {
|
|
|
107
168
|
status: ScenarioRunStatus;
|
|
108
169
|
/** The surface the run targeted: `default`, `browser`, … */
|
|
109
170
|
surface: string;
|
|
171
|
+
/** Absent on a run of the whole suite; see {@link ScenarioRunSelection}. */
|
|
172
|
+
selection?: ScenarioRunSelection;
|
|
110
173
|
startedAt: string;
|
|
111
174
|
finishedAt?: string;
|
|
112
175
|
}
|
|
@@ -166,11 +166,22 @@ export interface TestIdSelector {
|
|
|
166
166
|
* interface via `declare module`, so `wire.browser.page` is a fully typed
|
|
167
167
|
* Playwright `Page` in a project that installs it.
|
|
168
168
|
*/
|
|
169
|
+
/** How one deliberate screenshot is taken, and what it is for. */
|
|
170
|
+
export interface ScenarioScreenshotOptions {
|
|
171
|
+
/**
|
|
172
|
+
* Publish this one outside the run — a marketing card, a docs page, a
|
|
173
|
+
* gallery. Declared at the call site because only the author of the step
|
|
174
|
+
* knows the page is at a moment worth showing a stranger.
|
|
175
|
+
*/
|
|
176
|
+
showcase?: boolean;
|
|
177
|
+
/** Photograph the whole scrollable page rather than the viewport. */
|
|
178
|
+
fullPage?: boolean;
|
|
179
|
+
}
|
|
169
180
|
export interface PikkuBrowserWire {
|
|
170
181
|
/** The actor whose browser context this is */
|
|
171
182
|
readonly actor: string;
|
|
172
183
|
goto(url: string): Promise<void>;
|
|
173
|
-
screenshot(name?: string): Promise<Uint8Array>;
|
|
184
|
+
screenshot(name?: string, options?: ScenarioScreenshotOptions): Promise<Uint8Array>;
|
|
174
185
|
}
|
|
175
186
|
/**
|
|
176
187
|
* What one actor's window looked like at the moment a scenario failed.
|
|
@@ -224,6 +235,15 @@ export interface ScenarioBrowserProvider {
|
|
|
224
235
|
* scenario's reset, long after the outcome that decides whether to keep them.
|
|
225
236
|
*/
|
|
226
237
|
endScenario?(outcome: 'passed' | 'failed'): void;
|
|
238
|
+
/**
|
|
239
|
+
* When this actor's recording started, as epoch milliseconds.
|
|
240
|
+
*
|
|
241
|
+
* The seam that keeps the video clock out of `@pikku/core`: a driver knows
|
|
242
|
+
* when it opened the context it passed `recordVideo` to, and the runner turns
|
|
243
|
+
* that into a per-step offset. Absent for an actor with no window open, and
|
|
244
|
+
* for a run recording nothing — both of which leave the step's offset off.
|
|
245
|
+
*/
|
|
246
|
+
videoStartedAt?(actorName: string): number | undefined;
|
|
227
247
|
/**
|
|
228
248
|
* Snapshot every open window for a failed scenario. `label` identifies the
|
|
229
249
|
* scenario in artifact filenames. Never throws: a failure to capture must
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import type { CorePikkuFunctionConfig, CorePikkuFunctionHook } from '../../function/functions.types.js';
|
|
2
2
|
export type { ScenarioStepInvocation, ScenarioStepMeta, PikkuScenarioWire, } from './dsl/workflow-dsl.types.js';
|
|
3
|
-
export type { ScenarioStepPhase, ScenarioStepKind, ScenarioStepOptions, PikkuScenarioStepWire, ScenarioEnvironment, ScenarioSurface, ScenarioSurfaceResolution, PikkuBrowserWire, TestIdSelector, ScenarioBrowserProvider, ScenarioBrowserFailure, } from './scenario-step.types.js';
|
|
3
|
+
export type { ScenarioStepPhase, ScenarioStepKind, ScenarioStepOptions, PikkuScenarioStepWire, ScenarioEnvironment, ScenarioSurface, ScenarioSurfaceResolution, PikkuBrowserWire, ScenarioScreenshotOptions, TestIdSelector, ScenarioBrowserProvider, ScenarioBrowserFailure, } from './scenario-step.types.js';
|
|
4
4
|
export type CoreFeatureScenario = CorePikkuFunctionConfig<any, any, any> | {
|
|
5
5
|
scenario: CorePikkuFunctionConfig<any, any, any>;
|
|
6
6
|
data: unknown;
|
|
@@ -9,6 +9,14 @@ export type CoreFeature = {
|
|
|
9
9
|
name: string;
|
|
10
10
|
description?: string;
|
|
11
11
|
tags?: string[];
|
|
12
|
+
/**
|
|
13
|
+
* Whether this feature is guide material. Defaults to true: a feature is a
|
|
14
|
+
* page of the user guide unless it says otherwise, so a feature nobody has
|
|
15
|
+
* written about is a gap `pikku scenario guide` reports rather than a page
|
|
16
|
+
* silently missing. Pure plumbing — a wire, a validation layer, a bearer
|
|
17
|
+
* auth handshake — sets it false and stops being a coverage problem.
|
|
18
|
+
*/
|
|
19
|
+
document?: boolean;
|
|
12
20
|
scenarios: readonly CoreFeatureScenario[];
|
|
13
21
|
before?: CorePikkuFunctionHook;
|
|
14
22
|
after?: CorePikkuFunctionHook;
|
|
@@ -22,6 +30,8 @@ export type FeatureMeta = {
|
|
|
22
30
|
name: string;
|
|
23
31
|
description?: string;
|
|
24
32
|
tags: string[];
|
|
33
|
+
/** Present only when the feature opted out; absent means documented. */
|
|
34
|
+
document?: boolean;
|
|
25
35
|
entries: FeatureMetaEntry[];
|
|
26
36
|
unresolvedEntries: number;
|
|
27
37
|
hasBefore: boolean;
|
package/package.json
CHANGED