@pikku/inspector 0.12.56 → 0.12.59

package/CHANGELOG.md

@@ -1,3 +1,188 @@
+## 0.12.59
+
+### Patch Changes
+
+- 02c4fe5: fix(core,inspector): let a host grant an addon secrets it could not declare
+
+  Scoping an addon's `SecretService` to its `declaredSecrets` left generic addons
+  with nothing readable: `declaredSecrets` is derived from the addon package's own
+  source, but the secrets an addon like `@pikku/addon-graph` reads are named by the
+  consuming app's workflow nodes at runtime. Every authenticated `graph:httpRequest`
+  threw.
+
+  `wireAddon` now takes `secretGrants: string[]` and `credentialGrants: string[]`,
+  completing the grant family alongside `secretOverrides` (grant + rename) and
+  `globalSecrets` (grant everything, with a reason). Grants name the secret as the
+  addon reads it, since the scope check runs before the override map renames it —
+  which is also why an override's key grants and its value does not.
+
+  A grant naming a secret the project does not declare is an `INVALID_VALUE`
+  critical at codegen, resolved through the override map before lookup.
+
+- 438b776: Move the scenario and feature surface off `@pikku/core/workflow` and onto
+  `@pikku/core/scenario`. Scenarios extend workflows, so the production workflow
+  wiring no longer names a scenario module in its import graph. Feature and
+  scenario types are declared in their own `scenario.types.ts` rather than in
+  `workflow.types.ts`. Import `requireActor`, `requireScenarioEnv`, `pollUntil`,
+  `createCookieJar`, `addFeature`, `ScenarioHttpResponse` and the rest from
+  `@pikku/core/scenario`; `HttpPersonasConfig` now comes from
+  `@pikku/core/persona` rather than `@pikku/core/services`.
+- ad63f47: feat(cli): warn before codegen when a linked dependency splits a package's type identity
+
+  `pikku all` now runs the split-type-identity check as a preflight, beside the
+  existing `@pikku/core` one, and warns with `PKU719` naming each package, both
+  versions and both paths.
+
+  It has to run _before_ the work rather than after it fails. The failure it
+  explains is a V8 heap OOM, which aborts the process — `process.on('exit')`,
+  `uncaughtException` and `finally` never run, so nothing printed after the fact is
+  ever seen. By the time there is a symptom, the only thing that can help is
+  already on screen above it. Without this the user sees a codegen step that dies
+  of memory pressure with no indication that two copies of one package are the
+  reason, and the obvious next move is to raise `--max-old-space-size`, which
+  hides it further.
+
+  Warns rather than throws: a skewed linked dependency is a strong signal, not a
+  certainty, and refusing to build on a heuristic would break working setups.
+  `PIKKU_SKIP_TYPE_IDENTITY_CHECK=1` silences it, matching
+  `PIKKU_ALLOW_DUPLICATE_CORE`. It also swallows its own errors — it runs on every
+  codegen, so it must never be the reason a build stops.
+
+- Updated dependencies [02c4fe5]
+- Updated dependencies [438b776]
+- Updated dependencies [438b776]
+  - @pikku/core@0.12.83
+
+## 0.12.58
+
+### Patch Changes
+
+- 3ad2131: Name models by what they are for, and switch them all in one place
+
+  A `models` table in pikku.config.json maps an alias to a provider-qualified
+  model, so a declaration can say `model: 'cheap'` and the project repoints every
+  use of that tier at once instead of editing each agent. A model containing `/`
+  is still concrete and used exactly as written, which is how an agent that needs
+  one specific model pins it — aliases are opt-in.
+
+  The table is baked into codegen rather than read at runtime, so it applies to
+  deployed units and not just local runs, and `pikku dev`/`pikku serve` take
+  `--model cheap:openai/gpt-5-nano` to repoint a tier for one run without editing
+  the config.
+
+  Because the inspector already holds every agent's model literal, a bare name
+  with no matching alias now fails the build (PKU146) naming the aliases that do
+  exist, rather than reaching a provider as an unknown model.
+
+  Aliases resolve for every modality, not just agents: image, speech,
+  transcription, embedding and reranking all reach a provider through the same
+  point in the Vercel runner.
+
+- b930dca: Remove the `secretBroker` escape hatch and scope addon secrets and credentials
+
+  `secretBroker` let three named console functions receive the real `SecretService`,
+  against the rule that a function never sees one. It is gone: the inspector allowlist,
+  the `FunctionRuntimeMeta` flag, the runner branches, and the `WiredSecretBrokerServices`
+  type. Console secret administration moved into the console addon, where a
+  `SecretAdminService` holds the `SecretService` and the functions hold none.
+
+  Addons are now scoped rather than trusted. The CLI emits each package's declared secret
+  keys, and the host wraps the `SecretService` in a `ScopedSecretService` and the
+  `CredentialService` in a new `ScopedCredentialService` before the addon's service factory
+  runs — so an addon reads only what it declared, cannot write secrets, and cannot enumerate
+  the app's users. `wireAddon({ globalSecrets, globalCredentials })` waives this, taking the
+  reason as its value; only the consuming app can grant it, and the deploy manifest reports
+  every grant under `unscopedSecretAddons` / `unscopedCredentialAddons`.
+
+- 8978fbd: feat(workflow): let an approval gate declare who may answer it
+
+  `workflow.approval()` gains `approvers` (`'any' | 'owner' | 'not-initiator'`)
+  and `approverScope`, so a gate can require four-eyes sign-off, restrict itself
+  to the run's initiator, or require the decider to hold a named scope.
+
+  Both are enforced when the workflow replays the gate — the same place, and for
+  the same reason, the decision payload is validated: the policy is a value on
+  the workflow, and a decision can be recorded before the run has ever reached
+  the gate. A decision that fails the policy is discarded and the gate stays
+  closed. Where the run has already published its policy, the check also runs at
+  submission time so the caller gets a 403 rather than silence.
+
+  An answer is now recorded where it can be answered for later. The settled
+  decision carries `decidedBy` and `decidedAt` in its `ApprovalOutcome`, so who
+  signed reaches `workflowStep.result` and `workflowStepHistory` rather than
+  living only in mutable run state. Every answer — accepted, refused at the door,
+  or cleared on replay — is also written to the audit sink as
+  `workflow.approval.decided`, which outlives the run: `deleteRun` cascades to
+  steps and history, and a refused attempt never reaches a step at all. Projects
+  with no audit service wired are unaffected.
+
+  **This loosens the default.** `approveStep` previously refused anyone but the
+  run's initiator, unconditionally. A gate that declares no `approvers` now
+  accepts a decision from anyone the approve entrypoint admits — restore the old
+  behaviour per-gate with `approvers: 'owner'`, or gate the approve route with
+  `auth`/`permissions`. Ownership still governs _reads_ of a run unchanged.
+
+- Updated dependencies [063f43a]
+- Updated dependencies [ce66bf8]
+- Updated dependencies [d0307a8]
+- Updated dependencies [ce66bf8]
+- Updated dependencies [3ad2131]
+- Updated dependencies [b930dca]
+- Updated dependencies [b95e77d]
+- Updated dependencies [fd9d834]
+- Updated dependencies [8978fbd]
+  - @pikku/core@0.12.82
+
+## 0.12.57
+
+### Patch Changes
+
+- e110c55: Wire `pikkuAIScorer` / `pikkuAIJudge` through the inspector and codegen, and let an agent name the scorers that grade it.
+
+  The inspector reads a scorer's lane off which constructor was called rather than
+  off a field, so the two lanes cannot disagree with the code that produced them,
+  and refuses a scorer with no name or description — the meta is the only thing
+  that names it at runtime. An agent naming a scorer that was never declared is a
+  build error (`PKU155`) rather than an agent that quietly grades nothing forever.
+
+  Codegen emits a `ScorerName` union, so `scorers` on an agent is checked against
+  the scorers the project actually declares, plus the scorer wirings and meta.
+  `pikku validate` now also flags a scorer declared outside a `*.scorer.ts` file,
+  for the same reason scenarios have to live in files named for them: a rubric
+  buried in an agent definition is one nobody reviews as a rubric.
+
+- e110c55: Add `scenario.expectScore` — grade a finished agent run with a declared scorer and assert on it.
+
+  An agent's answer cannot be matched against a fixed string, so a scenario grades
+  it instead. `expectScore(step, runId, scorer, { atLeast, atMost, reference })`
+  runs one declared scorer against the run the scenario just triggered and fails
+  with the reason the judge gave. The default bound is `atLeast: 0.5`, so an
+  unqualified assertion still fails a run graded zero.
+
+  Grading goes over the new `pikkuScenarioGradeRun` instrumentation RPC, which the
+  dev server registers alongside the coverage and stub RPCs — so it exists only in
+  processes that should have it, and never in a deployed bundle. It grades from
+  the snapshot the runtime already took when the run finished, which is what makes
+  a scenario's grade the same measurement production's sampler makes rather than
+  an approximation of it: a run's prompt, answer and tool calls are spread across
+  a thread's messages, where the boundary of one run is not recoverable.
+
+  Two things differ deliberately from live scoring. The sample rate is ignored — a
+  scorer grading 1% of traffic still grades every scenario run — and the grade is
+  returned rather than recorded, so a test's score never lands among the
+  production figures. `reference` supplies the answer key a `requiresReference`
+  judge grades against, which is the only way such a judge is reachable at all.
+
+- Updated dependencies [e110c55]
+- Updated dependencies [e110c55]
+- Updated dependencies [e110c55]
+- Updated dependencies [acc8077]
+- Updated dependencies [905f737]
+- Updated dependencies [3cc6428]
+- Updated dependencies [c524adf]
+- Updated dependencies [e110c55]
+  - @pikku/core@0.12.81
+
 ## 0.12.56
 
 ### Patch Changes

package/dist/add/add-ai-agent.js

@@ -217,6 +217,29 @@ function resolveIdentifierToAgentName(identifier, checker) {
     }
     return null;
 }
+/**
+ * The scorers an agent asks to be graded by, which are plain names rather than
+ * references: a scorer is resolved out of the registry at grading time, not
+ * imported by the agent, so nothing here has a declaration to chase.
+ */
+function resolveScorerNames(obj, agentName, logger) {
+    const property = obj.properties.find((p) => ts.isPropertyAssignment(p) &&
+        ts.isIdentifier(p.name) &&
+        p.name.text === 'scorers');
+    if (!property || !ts.isPropertyAssignment(property))
+        return null;
+    if (!ts.isArrayLiteralExpression(property.initializer))
+        return null;
+    const names = [];
+    for (const element of property.initializer.elements) {
+        if (!ts.isStringLiteral(element)) {
+            logger.critical(ErrorCode.INVALID_VALUE, `AI agent '${agentName}' lists a scorer that is not an inline string: \`${element.getText()}\`. A scorer is named, not imported, so the name has to be readable from source.`);
+            continue;
+        }
+        names.push(element.text);
+    }
+    return names.length > 0 ? names : null;
+}
 export const addAIAgent = (logger, node, checker, state, options) => {
     if (!ts.isCallExpression(node)) {
         return;
@@ -244,6 +267,7 @@ export const addAIAgent = (logger, node, checker, state, options) => {
         const maxStepsValue = getPropertyValue(obj, 'maxSteps');
         const temperatureValue = getPropertyValue(obj, 'temperature');
         const toolChoiceValue = getPropertyValue(obj, 'toolChoice');
+        const scorersValue = resolveScorerNames(obj, nameValue || '', logger);
         const toolsValue = resolveToolReferences(obj, checker, nameValue || '', logger);
         if (toolsValue) {
             for (const toolName of toolsValue) {
@@ -389,6 +413,7 @@ export const addAIAgent = (logger, node, checker, state, options) => {
             ...(toolsValue !== null && { tools: toolsValue }),
             ...(agentsValue !== null && { agents: agentsValue }),
             ...(workflowsValue !== null && { workflows: workflowsValue }),
+            ...(scorersValue !== null && { scorers: scorersValue }),
             tags,
             inputSchema,
             outputSchema,

package/dist/add/add-ai-scorer.d.ts

@@ -0,0 +1,2 @@
+import type { AddWiring } from '../types.js';
+export declare const addAIScorer: AddWiring;

package/dist/add/add-ai-scorer.js

@@ -0,0 +1,61 @@
+import * as ts from 'typescript';
+import { getPropertyValue } from '../utils/get-property-value.js';
+import { extractFunctionName } from '../utils/extract-function-name.js';
+import { ErrorCode } from '../error-codes.js';
+/**
+ * `pikkuAIScorer` grades in code and `pikkuAIJudge` grades with a model, which
+ * is the whole difference between the two lanes — so the lane is read off which
+ * constructor was called rather than from a field an author could contradict.
+ */
+const LANES = {
+    pikkuAIScorer: 'fast',
+    pikkuAIJudge: 'slow',
+};
+export const addAIScorer = (logger, node, checker, state) => {
+    if (!ts.isCallExpression(node))
+        return;
+    const expression = node.expression;
+    if (!ts.isIdentifier(expression))
+        return;
+    const lane = LANES[expression.text];
+    if (!lane)
+        return;
+    const firstArg = node.arguments[0];
+    if (!firstArg || !ts.isObjectLiteralExpression(firstArg))
+        return;
+    const obj = firstArg;
+    const { exportedName } = extractFunctionName(node, checker, state.rootDir);
+    const nameValue = getPropertyValue(obj, 'name');
+    if (!nameValue) {
+        logger.critical(ErrorCode.MISSING_NAME, "AI scorer is missing the required 'name' property.");
+        return;
+    }
+    const description = getPropertyValue(obj, 'description');
+    if (!description) {
+        logger.critical(ErrorCode.MISSING_DESCRIPTION, `AI scorer '${nameValue}' is missing the required 'description' property.`);
+        return;
+    }
+    const sampleRateValue = getPropertyValue(obj, 'sampleRate');
+    if (sampleRateValue !== null &&
+        (sampleRateValue < 0 || sampleRateValue > 1)) {
+        logger.critical(ErrorCode.INVALID_VALUE, `AI scorer '${nameValue}' has a sampleRate of ${sampleRateValue} — it is the fraction of live runs to grade, so it must be between 0 and 1.`);
+        return;
+    }
+    const requiresReference = getPropertyValue(obj, 'requiresReference') ?? false;
+    const scorerKey = exportedName || nameValue;
+    if (exportedName) {
+        state.scorers.files.set(scorerKey, {
+            path: node.getSourceFile().fileName,
+            exportedName,
+        });
+    }
+    state.scorers.scorersMeta[scorerKey] = {
+        name: nameValue,
+        description,
+        lane,
+        sampleRate: sampleRateValue ?? 1,
+        requiresReference,
+        sourceFile: node.getSourceFile().fileName,
+        exportedName: exportedName || undefined,
+    };
+};

package/dist/add/add-wire-addon.js

@@ -53,6 +53,10 @@ export function addWireAddon(node, state, logger) {
     let secretOverrides;
     let variableOverrides;
     let credentialOverrides;
+    let secretGrants;
+    let credentialGrants;
+    let globalSecrets;
+    let globalCredentials;
     for (const prop of firstArg.properties) {
         if (!ts.isPropertyAssignment(prop) || !ts.isIdentifier(prop.name))
             continue;
@@ -94,6 +98,24 @@ export function addWireAddon(node, state, logger) {
             ts.isObjectLiteralExpression(prop.initializer)) {
             credentialOverrides = parseStringRecord(prop.initializer);
         }
+        else if (key === 'secretGrants') {
+            secretGrants = parseStringArray(prop.initializer);
+        }
+        else if (key === 'credentialGrants') {
+            credentialGrants = parseStringArray(prop.initializer);
+        }
+        else if (key === 'globalSecrets') {
+            // The grant is real at runtime whatever the reason evaluates to, so a
+            // non-literal is recorded as its source text rather than dropped.
+            globalSecrets = ts.isStringLiteral(prop.initializer)
+                ? prop.initializer.text
+                : prop.initializer.getText();
+        }
+        else if (key === 'globalCredentials') {
+            globalCredentials = ts.isStringLiteral(prop.initializer)
+                ? prop.initializer.text
+                : prop.initializer.getText();
+        }
     }
     if (!name || !pkg)
         return;
@@ -108,6 +130,10 @@ export function addWireAddon(node, state, logger) {
         secretOverrides,
         variableOverrides,
         credentialOverrides,
+        secretGrants,
+        credentialGrants,
+        globalSecrets,
+        globalCredentials,
     });
     state.rpc.usedAddons.add(name);
     state.rpc.wireAddonFiles.add(node.getSourceFile().fileName);

package/dist/add/scenario-instrumentation.d.ts

@@ -13,8 +13,13 @@
  * `pikku dev` registers the implementations itself. Ignoring by name (rather
  * than deleting the scaffold and requiring every project to regenerate) is what
  * keeps an existing project's checked-in `scenarios.gen.ts` from deploying.
+ *
+ * `pikkuScenarioGradeRun` was never scaffolded — it arrived after the split —
+ * but it belongs here for the same reason: grading a run on demand, ignoring the
+ * sample rate that governs live traffic, is a thing you do to a development
+ * server, and the name is reserved so an application cannot claim it.
  */
-export declare const SCENARIO_INSTRUMENTATION_FUNCTIONS: readonly ["pikkuScenarioTakeLiveCoverage", "pikkuScenarioResetLiveCoverage", "pikkuScenarioResetStubs", "pikkuScenarioGetStubCalls"];
+export declare const SCENARIO_INSTRUMENTATION_FUNCTIONS: readonly ["pikkuScenarioTakeLiveCoverage", "pikkuScenarioResetLiveCoverage", "pikkuScenarioResetStubs", "pikkuScenarioGetStubCalls", "pikkuScenarioGradeRun"];
 export type ScenarioInstrumentationFunction = (typeof SCENARIO_INSTRUMENTATION_FUNCTIONS)[number];
 /**
  * True for a scenario instrumentation function, by name. Versioned ids

package/dist/add/scenario-instrumentation.js

@@ -13,12 +13,18 @@
  * `pikku dev` registers the implementations itself. Ignoring by name (rather
  * than deleting the scaffold and requiring every project to regenerate) is what
  * keeps an existing project's checked-in `scenarios.gen.ts` from deploying.
+ *
+ * `pikkuScenarioGradeRun` was never scaffolded — it arrived after the split —
+ * but it belongs here for the same reason: grading a run on demand, ignoring the
+ * sample rate that governs live traffic, is a thing you do to a development
+ * server, and the name is reserved so an application cannot claim it.
  */
 export const SCENARIO_INSTRUMENTATION_FUNCTIONS = [
     'pikkuScenarioTakeLiveCoverage',
     'pikkuScenarioResetLiveCoverage',
     'pikkuScenarioResetStubs',
     'pikkuScenarioGetStubCalls',
+    'pikkuScenarioGradeRun',
 ];
 const names = new Set(SCENARIO_INSTRUMENTATION_FUNCTIONS);
 /**

package/dist/error-codes.d.ts

@@ -62,12 +62,14 @@ export declare enum ErrorCode {
     AGENT_TOOL_UNKNOWN_NAMESPACE = "PKU152",
     AGENT_TOOL_NOT_FOUND = "PKU153",
     AGENT_TOOL_MISSING_DESCRIPTION = "PKU154",
+    AGENT_SCORER_NOT_FOUND = "PKU155",
     SCHEMA_AND_WIRING_COLOCATED = "PKU490",
     SERVICES_NOT_DESTRUCTURED = "PKU410",
     WIRES_NOT_DESTRUCTURED = "PKU411",
     FUNCTION_DYNAMIC_IMPORT = "PKU498",
     DUPLICATE_CORE_VERSION = "PKU717",
     CORE_VERSION_SKEW = "PKU718",
+    SPLIT_TYPE_IDENTITY = "PKU719",
     PII_IN_OUTPUT = "PKU910",
     ADDON_WIRING_NOT_ALLOWED = "PKU920",
     ADDON_CONTRACT_HANDLERS_NOT_ALLOWED = "PKU921",

package/dist/error-codes.js

@@ -79,6 +79,8 @@ export var ErrorCode;
     ErrorCode["AGENT_TOOL_UNKNOWN_NAMESPACE"] = "PKU152";
     ErrorCode["AGENT_TOOL_NOT_FOUND"] = "PKU153";
     ErrorCode["AGENT_TOOL_MISSING_DESCRIPTION"] = "PKU154";
+    // Agent scorer reference errors
+    ErrorCode["AGENT_SCORER_NOT_FOUND"] = "PKU155";
     // File structure errors
     ErrorCode["SCHEMA_AND_WIRING_COLOCATED"] = "PKU490";
     // Optimization diagnostics
@@ -88,6 +90,7 @@ export var ErrorCode;
     // Dependency integrity errors
     ErrorCode["DUPLICATE_CORE_VERSION"] = "PKU717";
     ErrorCode["CORE_VERSION_SKEW"] = "PKU718";
+    ErrorCode["SPLIT_TYPE_IDENTITY"] = "PKU719";
     // Data classification errors
     ErrorCode["PII_IN_OUTPUT"] = "PKU910";
     // Addon authoring errors

package/dist/inspector.js

@@ -6,7 +6,7 @@ import { TypesMap } from './types-map.js';
 import { getFilesAndMethods } from './utils/get-files-and-methods.js';
 import { findCommonAncestor } from './utils/find-root-dir.js';
 import { createNestedProjectFilter } from './utils/nested-project-filter.js';
-import { aggregateRequiredServices, stampAuthHandlerServices, validateAgentModels, validateAgentToolReferences, validateSecretOverrides, validateVariableOverrides, validateCredentialOverrides, validateRemoteAddonDependencies, validateRemoteAddonAuth, validateScopeReferences, validateSystemRoleScopes, validatePersonaRoles, computeResolvedIOTypes, computeMiddlewareGroupsMeta, computePermissionsGroupsMeta, computeRequiredSchemas, computeSecretBrokers, validateNoSecretAliasServices, validateSecretUsage, computeDiagnostics, validateSchemaReferences, validateSchemaWiringSeparation, validateScenarioServices, validateScenarioSteps, validateWorkflowGraphAddons, } from './utils/post-process.js';
+import { aggregateRequiredServices, stampAuthHandlerServices, validateAgentModels, validateAgentToolReferences, validateAgentScorerReferences, validateSecretOverrides, validateVariableOverrides, validateCredentialOverrides, validateRemoteAddonDependencies, validateRemoteAddonAuth, validateScopeReferences, validateSystemRoleScopes, validatePersonaRoles, computeResolvedIOTypes, computeMiddlewareGroupsMeta, computePermissionsGroupsMeta, computeRequiredSchemas, validateNoSecretAliasServices, validateSecretUsage, computeDiagnostics, validateSchemaReferences, validateSchemaWiringSeparation, validateScenarioServices, validateScenarioSteps, validateWorkflowGraphAddons, } from './utils/post-process.js';
 import { validateExposedFunctionsGated } from './utils/validate-exposed-functions-gated.js';
 import { validateTagsResolveToMiddleware } from './utils/validate-tags-resolve-to-middleware.js';
 import { annotateHttpRouteAuth } from './utils/annotate-http-route-auth.js';
@@ -116,6 +116,10 @@ export function getInitialInspectorState(rootDir) {
             agentsMeta: {},
             files: new Map(),
         },
+        scorers: {
+            scorersMeta: {},
+            files: new Map(),
+        },
         cli: {
             meta: {
                 programs: {},
@@ -366,7 +370,6 @@ export const inspect = async (logger, routeFiles, options = {}) => {
         computeResolvedIOTypes(state);
         computeMiddlewareGroupsMeta(state);
         computePermissionsGroupsMeta(state);
-        computeSecretBrokers(state);
         validateNoSecretAliasServices(logger, checker, state);
         validateSecretUsage(logger, state);
         computeDiagnostics(state);
@@ -384,8 +387,9 @@ export const inspect = async (logger, routeFiles, options = {}) => {
         if (options.openAPI) {
             state.openAPISpec = await generateOpenAPISpec(logger, state.functions.meta, state.http.meta, state.schemas, options.openAPI.additionalInfo, pikkuState(null, 'misc', 'errors'));
         }
-        validateAgentModels(logger, state);
+        validateAgentModels(logger, state, options.modelAliases);
         validateAgentToolReferences(logger, state, options);
+        validateAgentScorerReferences(logger, state);
         validateSecretOverrides(logger, state);
         validateVariableOverrides(logger, state);
         validateCredentialOverrides(logger, state);

package/dist/types.d.ts

@@ -9,6 +9,7 @@ import type { QueueWorkersMeta } from '@pikku/core/queue';
 import type { WorkflowsMeta } from '@pikku/core/workflow';
 import type { MCPResourceMeta, MCPToolMeta, MCPPromptMeta } from '@pikku/core/mcp';
 import type { AIAgentMeta } from '@pikku/core/ai-agent';
+import type { ScorerMeta } from '@pikku/core/ai-scorer';
 import type { CLIMeta } from '@pikku/core/cli';
 import type { CLICommandMeta } from '@pikku/core/cli';
 import type { NodesMeta } from '@pikku/core/node';
@@ -229,6 +230,8 @@ export type InspectorOptions = Partial<{
      * tool's own name, so this is a quality gate rather than a correctness one.
      */
     strictMeta: boolean;
+    /** The `models` table from pikku.config.json, alias -> `provider/model`. */
+    modelAliases: Record<string, string>;
     sourceFile: ts.SourceFile;
     /**
      * The project's tsconfig, read for its path mappings only. The inspector's
@@ -518,6 +521,20 @@ export interface InspectorState {
             secretOverrides?: Record<string, string>;
             variableOverrides?: Record<string, string>;
             credentialOverrides?: Record<string, string>;
+            /** Secrets the app lends this instance, named as the addon reads them. */
+            secretGrants?: string[];
+            /** Credentials the app lends this instance, named as the addon reads them. */
+            credentialGrants?: string[];
+            /**
+             * The app's stated reason for handing this instance the whole
+             * `SecretService` instead of one scoped to its declared secrets.
+             */
+            globalSecrets?: string;
+            /**
+             * The app's stated reason for handing this instance the whole
+             * `CredentialService` instead of one scoped to its declared credentials.
+             */
+            globalCredentials?: string;
         }>;
         wireAddonFiles: Set<string>;
     };
@@ -534,6 +551,13 @@ export interface InspectorState {
             exportedName: string;
         }>;
     };
+    scorers: {
+        scorersMeta: ScorerMeta;
+        files: Map<string, {
+            path: string;
+            exportedName: string;
+        }>;
+    };
     cli: {
         meta: CLIMeta;
         files: Set<string>;

package/dist/utils/filter-inspector-state.js

@@ -328,6 +328,14 @@ export function filterInspectorState(state, filters, logger) {
             agentsMeta: JSON.parse(JSON.stringify(state.agents?.agentsMeta ?? {})),
             files: new Map(),
         },
+        // Scorers carry through every filter. They are named by the agents that
+        // survive it, and a deployment that quietly stopped grading would look
+        // exactly like one whose scorers all returned nothing.
+        scorers: {
+            ...state.scorers,
+            scorersMeta: JSON.parse(JSON.stringify(state.scorers?.scorersMeta ?? {})),
+            files: new Map(state.scorers?.files ?? []),
+        },
         rpc: {
             ...state.rpc,
             internalMeta: { ...state.rpc.internalMeta }, // Clone to avoid mutating original

package/dist/utils/post-process.d.ts

@@ -65,7 +65,11 @@ export declare function computeRequiredSchemas(state: InspectorState, options: I
  * `--fail-on-error` without failing a dev server over a function nobody calls.
  */
 export declare function validateSchemaReferences(logger: InspectorLogger, state: InspectorState): void;
-export declare function validateAgentModels(logger: InspectorLogger, state: InspectorState | Omit<InspectorState, 'typesLookup'>): void;
+/**
+ * A model is either concrete (`provider/model`) or an alias, which must appear
+ * in the `models` table from pikku.config.json.
+ */
+export declare function validateAgentModels(logger: InspectorLogger, state: InspectorState | Omit<InspectorState, 'typesLookup'>, modelAliases?: Record<string, string>): void;
 /**
  * Scenarios are pure stories of remote RPCs (same rule as client-side CLI
  * renderers): the func may only destructure logger/config — everything else
@@ -111,8 +115,6 @@ export declare function validateWorkflowGraphAddons(logger: InspectorLogger, sta
  * because the pikku state metadata doesn't exist in the CLI context.
  */
 export declare function validateSchemaWiringSeparation(logger: InspectorLogger, state: InspectorState): void;
-/** Marks the functions that keep the full `SecretService`. */
-export declare function computeSecretBrokers(state: InspectorState): void;
 /**
  * Refuses a singleton service that is a `SecretService` under another name.
  *
@@ -156,6 +158,14 @@ export declare function validateScopeReferences(logger: InspectorLogger, state:
  * a title labels a tool in a UI, it does not tell a model when to reach for it.
  */
 export declare function validateAgentToolReferences(logger: InspectorLogger, state: InspectorState | Omit<InspectorState, 'typesLookup'>, options?: InspectorOptions): void;
+/**
+ * Validates that every scorer an agent asks to be graded by exists.
+ *
+ * The runtime warns and skips an unresolvable name rather than failing a run
+ * that has already answered the user — which means a typo would otherwise cost
+ * nothing visible and simply grade nothing, forever.
+ */
+export declare function validateAgentScorerReferences(logger: InspectorLogger, state: InspectorState | Omit<InspectorState, 'typesLookup'>): void;
 /**
  * Validates that every scope granted by a system role is declared via
  * `defineScope`. Runs after all visitors, so declaration order does not matter.