@pikku/inspector 0.12.85 → 0.12.86

package/CHANGELOG.md

@@ -1,3 +1,50 @@
+## 0.12.86
+
+### Patch Changes
+
+- f05fb4e: A scenario that belongs to no feature is now reported.
+
+  Features are what the guide is written against: prose cites a feature, and the compiler fills that citation with the evidence from the scenarios the feature owns. A scenario nobody listed is therefore invisible — it runs, it captures screenshots and recordings, and none of it ever reaches a page. That failure is silent today, and it is the single reason two real projects have hundreds of captures bound to nothing.
+
+  The inspector now walks every feature's scenario list, and any scenario not claimed by one is reported as `PKU682`. It is an error rather than a critical, so `pikku all` keeps passing by default and the list is there to work through; `--fail-on-error` makes it a wall once a project has caught up.
+
+  Membership reads the scenario each entry names, not the entry itself. A `data` the AST cannot evaluate — `'a'.repeat(64)`, a call, anything computed — leaves the entry partial for the console but says nothing about who owns the scenario, and the check no longer treats the two as the same thing.
+
+  An entry that names no scenario at all — a spread, a `.map()` — does leave membership unknown. That only matters once something is already unowned, so the check reports it then, as a warn naming both the unowned scenarios and the features that could be hiding them, rather than accusing a scenario that may well be in one of those arrays.
+
+- b4a895e: `pikku scenario guide` writes the user guide a scenario suite already contains. A feature reads as a page and a scenario as a section, and a run leaves screenshots behind with the captions their author took them under — the command joins that to the editorial prose a project checks in under `docs/` and writes markdown. It renders no HTML, ships no components, resolves no asset URLs and calls no model: an image is an ordinary relative `![caption](path)`, and whoever consumes the markdown rewrites the paths.
+
+  A page cites a feature by leaving the marker pair where the block belongs:
+
+  ```markdown
+  ---
+  title: Deployments
+  ---
+
+  A deployment is one tracked shipment of your app.
+
+  <!-- pikku:guide feature=deploymentsFeature -->
+  <!-- /pikku:guide -->
+
+  ## Does my app go down during a deploy?
+  ```
+
+  That one line does both halves of the job. It says _where_ the block goes, which a frontmatter list cannot express, and it is what the coverage gate counts to decide _whether_ a feature is documented at all. The mapping stays many-to-many and falls out of the union of every marker in the tree. A rebuild rewrites exactly the regions between the markers, so every sentence a human wrote around them survives.
+
+  **The steps are evidence, not content.** A generated block is the scenario's title, the description its author wrote, and the shots it filed — never a numbered Given/When/Then ladder, which is a test report and not something anybody arrives at a documentation page wanting. The sentences are still what the guide is kept honest against: `docs/.guide.lock` records a hash of each feature's step sentences and artifact ids, deliberately not of the image bytes. Restyling a UI changes every screenshot and no sentence; inserting or renaming a step changes what the prose was describing, and the page is reported stale. The lock is generated and checked in, so no hash is ever typed or merged by hand — and a tree whose lock is untracked reports every page as current forever.
+
+  Every registered feature has to be cited by some page, and a feature that is pure plumbing says so rather than being written about — `pikkuFeature({ document: false })`, threaded through the inspector and `FeatureMeta`. An uncited feature fails the command by name; `--allow-undocumented` downgrades that one failure to a report. A page citing a feature id that is not registered stays an error either way.
+
+  A guide is only written out of a run that can stand behind it. A run that failed or was killed halfway is refused, because a page is a claim that the product does what it says. So is a narrowed one: `pikku scenario run --flows`/`--features`/`--tags` leaves out scenarios the suite has, and a guide built from it would describe those flows as though they do not exist. `ScenarioRunRecord.selection` records the filters a run was selected with, since nothing in the results afterwards can tell a suite of forty from forty that were asked for.
+
+  Results are joined to features by `featureId`, falling back to the display name only for records written before that field existed — a title is rewritten freely and two features may share one.
+
+  Emission is deterministic — identical inputs give byte-identical output, and no timestamp goes in that did not come from the run record. Frontmatter the compiler does not own (`slug`, `draft`, `sidebar_position`, anything else a docs site reads) passes through untouched, and a source written with CRLF line endings is read as having frontmatter.
+
+- Updated dependencies [b4a895e]
+- Updated dependencies [b4a895e]
+  - @pikku/core@0.12.118
+
 ## 0.12.85
 
 ### Patch Changes

package/dist/add/add-feature.js

@@ -96,6 +96,16 @@ const stringProperty = (config, name) => {
     const value = getProperty(config, name);
     return value && ts.isStringLiteralLike(value) ? value.text : undefined;
 };
+const booleanProperty = (config, name) => {
+    const value = getProperty(config, name);
+    if (!value) {
+        return undefined;
+    }
+    if (value.kind === ts.SyntaxKind.TrueKeyword) {
+        return true;
+    }
+    return value.kind === ts.SyntaxKind.FalseKeyword ? false : undefined;
+};
 const stringArrayProperty = (config, name) => {
     const value = getProperty(config, name);
     if (!value || !ts.isArrayLiteralExpression(value)) {
@@ -106,6 +116,16 @@ const stringArrayProperty = (config, name) => {
         .map((e) => e.text);
     return values.length === value.elements.length ? values : undefined;
 };
+const mentionedScenario = (element, checker) => {
+    if (ts.isIdentifier(element)) {
+        return scenarioName(element, checker);
+    }
+    if (ts.isObjectLiteralExpression(element)) {
+        const reference = getProperty(element, 'scenario');
+        return reference ? scenarioName(reference, checker) : undefined;
+    }
+    return undefined;
+};
 const readEntry = (element, checker) => {
     if (ts.isIdentifier(element)) {
         const scenario = scenarioName(element, checker);
@@ -164,6 +184,8 @@ export const addFeature = (logger, node, checker, state) => {
         exportedName,
         entries: [],
         unresolvedEntries: 0,
+        mentions: [],
+        unnamedEntries: 0,
         hasBefore: false,
         hasAfter: false,
     };
@@ -172,6 +194,10 @@ export const addFeature = (logger, node, checker, state) => {
         const name = stringProperty(config, 'name');
         const description = stringProperty(config, 'description');
         const tags = stringArrayProperty(config, 'tags');
+        const document = booleanProperty(config, 'document');
+        if (document !== undefined) {
+            feature.document = document;
+        }
         if (name !== undefined) {
             feature.name = name;
         }
@@ -186,6 +212,13 @@ export const addFeature = (logger, node, checker, state) => {
         const scenarios = getProperty(config, 'scenarios');
         if (scenarios && ts.isArrayLiteralExpression(scenarios)) {
             for (const element of scenarios.elements) {
+                const mention = mentionedScenario(element, checker);
+                if (mention) {
+                    feature.mentions.push(mention);
+                }
+                else {
+                    feature.unnamedEntries += 1;
+                }
                 const entry = readEntry(element, checker);
                 if (entry) {
                     feature.entries.push(entry);

package/dist/error-codes.d.ts

@@ -25,6 +25,7 @@ export declare enum ErrorCode {
     SCENARIO_NOT_EXTRACTABLE = "PKU679",
     SCENARIO_HAS_NO_ASSERTION = "PKU680",
     SCENARIO_STEP_PROSE_NAMES_ACTOR = "PKU681",
+    SCENARIO_HAS_NO_FEATURE = "PKU682",
     WORKFLOW_ORCHESTRATOR_NOT_CONFIGURED = "PKU600",
     INVALID_DSL_WORKFLOW = "PKU641",
     WORKFLOW_GRAPH_ADDON_NOT_WIRED = "PKU642",

package/dist/error-codes.js

@@ -27,6 +27,7 @@ export var ErrorCode;
     ErrorCode["SCENARIO_NOT_EXTRACTABLE"] = "PKU679";
     ErrorCode["SCENARIO_HAS_NO_ASSERTION"] = "PKU680";
     ErrorCode["SCENARIO_STEP_PROSE_NAMES_ACTOR"] = "PKU681";
+    ErrorCode["SCENARIO_HAS_NO_FEATURE"] = "PKU682";
     ErrorCode["WORKFLOW_ORCHESTRATOR_NOT_CONFIGURED"] = "PKU600";
     ErrorCode["INVALID_DSL_WORKFLOW"] = "PKU641";
     ErrorCode["WORKFLOW_GRAPH_ADDON_NOT_WIRED"] = "PKU642";

package/dist/inspector.js

@@ -7,7 +7,7 @@ import { getFilesAndMethods } from './utils/get-files-and-methods.js';
 import { findCommonAncestor } from './utils/find-root-dir.js';
 import { createSourceFileCacheHost } from './source-file-cache.js';
 import { createNestedProjectFilter } from './utils/nested-project-filter.js';
-import { aggregateRequiredServices, stampAuthHandlerServices, validateAgentModels, validateAgentToolReferences, validateAgentScorerReferences, validateSecretOverrides, validateVariableOverrides, validateCredentialOverrides, validateRemoteAddonDependencies, validateRemoteAddonAuth, validateScopeReferences, validateSystemRoleScopes, validatePersonaRoles, computeResolvedIOTypes, computeMiddlewareGroupsMeta, computePermissionsGroupsMeta, computeRequiredSchemas, validateNoSecretAliasServices, registerDerivedOAuth2AppSecrets, validateSecretUsage, computeDiagnostics, validateSchemaReferences, validateSchemaWiringSeparation, validateScenarioServices, validateScenarioSteps, validateWorkflowGraphAddons, } from './utils/post-process.js';
+import { aggregateRequiredServices, stampAuthHandlerServices, validateAgentModels, validateAgentToolReferences, validateAgentScorerReferences, validateSecretOverrides, validateVariableOverrides, validateCredentialOverrides, validateRemoteAddonDependencies, validateRemoteAddonAuth, validateScopeReferences, validateSystemRoleScopes, validatePersonaRoles, computeResolvedIOTypes, computeMiddlewareGroupsMeta, computePermissionsGroupsMeta, computeRequiredSchemas, validateNoSecretAliasServices, registerDerivedOAuth2AppSecrets, validateSecretUsage, computeDiagnostics, validateSchemaReferences, validateSchemaWiringSeparation, validateScenarioServices, validateScenarioSteps, validateScenarioFeatures, validateWorkflowGraphAddons, } from './utils/post-process.js';
 import { validateExposedFunctionsGated } from './utils/validate-exposed-functions-gated.js';
 import { validateTagsResolveToMiddleware } from './utils/validate-tags-resolve-to-middleware.js';
 import { annotateHttpRouteAuth } from './utils/annotate-http-route-auth.js';
@@ -405,6 +405,7 @@ export const inspect = async (logger, routeFiles, options = {}) => {
         validateWorkflowGraphAddons(logger, state);
         validateScenarioServices(logger, state);
         validateScenarioSteps(logger, state);
+        validateScenarioFeatures(logger, state);
         // Needs every route, every function and every wireAddon declaration
         // together, so it can only run once the whole program is inspected.
         annotateHttpRouteAuth(state);

package/dist/types.d.ts

@@ -455,6 +455,8 @@ export interface InspectorFeature {
     name?: string;
     description?: string;
     tags?: string[];
+    /** `document: false` on the config, and only that — an absent flag documents. */
+    document?: boolean;
     entries: InspectorFeatureEntry[];
     /**
      * Entries that are not literal — a spread, a `.map()`, a call. Counted rather
@@ -462,6 +464,17 @@ export interface InspectorFeature {
      * quietly showing a short feature.
      */
     unresolvedEntries: number;
+    /**
+     * Every scenario the array names, including entries whose `data` could not be
+     * read. Membership is knowable there even when the entry is not — which is
+     * what the check for a scenario belonging to no feature compares against.
+     */
+    mentions: string[];
+    /**
+     * Entries that name no scenario at all — a spread, a `.map()`. Only these
+     * leave membership genuinely unknown.
+     */
+    unnamedEntries: number;
     /**
      * Feature hooks run once around the whole group, never per scenario, and are
      * runtime-only — so only their presence is recorded.

package/dist/utils/post-process.d.ts

@@ -129,6 +129,31 @@ export declare const proseOpensWithActor: (prose: string, actor: string) => bool
  * singletons the step actually needs.
  */
 export declare function validateScenarioSteps(logger: InspectorLogger, state: InspectorState | Omit<InspectorState, 'typesLookup'>): void;
+/**
+ * Reports scenarios that no `pikkuFeature` lists.
+ *
+ * A feature is what a run stamps onto each result, and the only handle anything
+ * downstream has on a scenario: the console groups by it and the guide cites it.
+ * A scenario in no feature still runs and still passes — it simply files its
+ * results, and any screenshot or recording it took, where nothing can reach
+ * them. That is not a style violation, it is work being thrown away, and it is
+ * invisible precisely because the suite is green.
+ *
+ * It is an `error` rather than a `critical` because every suite written before
+ * the rule has some, and a hard failure on a CLI bump would strand them with no
+ * way to defer. `--fail-on-error` is how a project opts in once it has bound
+ * what it has.
+ *
+ * Membership reads the scenario each entry names, not the entry itself: a
+ * `data` the AST cannot evaluate leaves the entry partial for the console but
+ * says nothing about who owns the scenario.
+ *
+ * An entry that names no scenario at all (a spread, a `.map()`) does leave
+ * membership unknown. That only matters when something is already unowned, so
+ * the check reports it then and downgrades to a warn naming both sides, rather
+ * than accusing a scenario that may well be in that array.
+ */
+export declare function validateScenarioFeatures(logger: InspectorLogger, state: InspectorState | Omit<InspectorState, 'typesLookup'>): void;
 /**
  * A `pikkuWorkflowGraph` node that references the `graph:` namespace (e.g.
  * `graph:editFields`) needs @pikku/addon-graph wired — otherwise the RPC never

package/dist/utils/post-process.js

@@ -943,6 +943,79 @@ export function validateScenarioSteps(logger, state) {
         }
     }
 }
+/**
+ * Reports scenarios that no `pikkuFeature` lists.
+ *
+ * A feature is what a run stamps onto each result, and the only handle anything
+ * downstream has on a scenario: the console groups by it and the guide cites it.
+ * A scenario in no feature still runs and still passes — it simply files its
+ * results, and any screenshot or recording it took, where nothing can reach
+ * them. That is not a style violation, it is work being thrown away, and it is
+ * invisible precisely because the suite is green.
+ *
+ * It is an `error` rather than a `critical` because every suite written before
+ * the rule has some, and a hard failure on a CLI bump would strand them with no
+ * way to defer. `--fail-on-error` is how a project opts in once it has bound
+ * what it has.
+ *
+ * Membership reads the scenario each entry names, not the entry itself: a
+ * `data` the AST cannot evaluate leaves the entry partial for the console but
+ * says nothing about who owns the scenario.
+ *
+ * An entry that names no scenario at all (a spread, a `.map()`) does leave
+ * membership unknown. That only matters when something is already unowned, so
+ * the check reports it then and downgrades to a warn naming both sides, rather
+ * than accusing a scenario that may well be in that array.
+ */
+export function validateScenarioFeatures(logger, state) {
+    const features = state.workflows.featureFiles;
+    if (!features) {
+        return;
+    }
+    const owned = new Set();
+    for (const feature of features.values()) {
+        for (const mention of feature.mentions) {
+            owned.add(mention);
+        }
+    }
+    const files = state.workflows.files;
+    const unowned = [];
+    for (const [workflowName, meta] of Object.entries(state.workflows.meta)) {
+        if (!meta.scenario) {
+            continue;
+        }
+        const exported = files?.get(workflowName)?.exportedName;
+        if (owned.has(workflowName) || (exported && owned.has(exported))) {
+            continue;
+        }
+        unowned.push(workflowName);
+    }
+    if (unowned.length === 0) {
+        return;
+    }
+    const unreadable = [...features.values()]
+        .filter((feature) => feature.unnamedEntries > 0)
+        .map((feature) => feature.exportedName);
+    if (unreadable.length > 0) {
+        logger.diagnostic({
+            severity: 'warn',
+            code: ErrorCode.SCENARIO_HAS_NO_FEATURE,
+            message: `${unowned.map((name) => `'${name}'`).join(', ')} appear in no feature's \`scenarios\` array, ` +
+                `but ${unreadable.map((name) => `'${name}'`).join(', ')} build theirs with a spread or a call, so one of them may still be listed there. ` +
+                `List those scenarios literally and this becomes a definite answer either way.`,
+        });
+        return;
+    }
+    for (const name of unowned) {
+        logger.diagnostic({
+            severity: 'error',
+            code: ErrorCode.SCENARIO_HAS_NO_FEATURE,
+            message: `Scenario '${name}' is listed by no pikkuFeature, so nothing downstream can reach it — ` +
+                `a run stamps no feature onto its results, the console groups it under none, and a guide cannot cite it. ` +
+                `Add it to a feature's \`scenarios\` array.`,
+        });
+    }
+}
 /**
  * A `pikkuWorkflowGraph` node that references the `graph:` namespace (e.g.
  * `graph:editFields`) needs @pikku/addon-graph wired — otherwise the RPC never

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pikku/inspector",
-  "version": "0.12.85",
+  "version": "0.12.86",
   "repository": {
     "type": "git",
     "url": "git+https://github.com/pikkujs/pikku.git",
@@ -55,10 +55,10 @@
     "zod-to-ts": "^2.1.0"
   },
   "peerDependencies": {
-    "@pikku/core": "^0.12.116"
+    "@pikku/core": "^0.12.118"
   },
   "devDependencies": {
-    "@pikku/core": "^0.12.116",
+    "@pikku/core": "^0.12.118",
     "@types/node": "^24.13.3"
   },
   "engines": {